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ABSTRACT : 

The kinematic aspect of surveillance-evasion is studied with a 
deterministic differential game model. The model considers a Pursuer 
with limitations on both speed and maneuverability (turning radius) 
and an Evader with only a speed limitation. Conditions are developed 
for the Pursuer to be able to maintain contact indefinitely. The 
results of this research modify previously published results on this 
problem. Shortcomings of previous work are discussed including the 
fact that the surveillance-evasion problem has not been solved for an 
arbitrary detection region. Related parts of the solution to Isaacs' 
homicidal chauffeur game and its one-sided counterpart are developed 
as background material. Some known allocation of effort in search 
theory results are derived by the Pontryagin maximum principle. 

This work was supported by The Office of Naval Research. 
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I. Introduction 



This report documents research findings for the time period 30 
March 1970 to 19 June 1970 under support of NR 276-027. This report 
discusses applications of the theory of differential games to pursuit 
and evasion problems of Naval warfare. In particular, we consider 
the problem of surveillance-evasion. A companion report [47] discusses 
other research findings of the contract period with respect to tactical 
allocation problems. 

The goal of this research is to determine the circumstances 
under which an evader can outmaneuver a pursuer to escape as a function 
of maneuverability. The solution of this problem leads to conditions 
for a tracker (destroyer) to be able to keep a hostile vehicle (sub- 
marine) under constant surveillance. The original approach was to 
survey the previously published work in this field and to attempt to 
extend these modelling efforts. Detailed analysis of past work [14] 
has uncovered several flaws in its mathematical development, and, 
hence, the current work has concentrated on establishing a firmer 
mathematical basis for the surveillance-evasion aspect of the more 
general problem of pursuit and evasion. This work has created a broad 
base for future possible extensions. 

Warfare is characterized by decisions being made on the dynamics 
of combat over a period of time by the antagonists towards conflicting 
goals. The creation of game theory by J. von Neumann [48], [49] 
(although anticipated by E. Borel [21]) has had a major impact on the 
modelling of conflict situations. The optimization of dynamical systems 
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has been studied under the calculus of variations since the 17th 
century. However, in cases where inequality constraints are present 
in the model, these powerful classical techniques require intricate 
modifications. In this environment, the almost simultaneous develop- 
ment in the early 1950's of differential games by R. Isaacs [26], [27], 
[28], [29] and the Pontryagin maximum principle by the Russian mathe- 
matician L. Pontryagin [43] has been enthusiastically received by 
military operations research workers. It seems appropriate to discuss 
these techniques briefly. 

a. Differential Games 

R. Isaacs was the original developer of differential games in 
the environment of RAND in the early 1950's [26], [27], [28], [29]. 

Although not acknowledged in his book, he applied Bellman 1 s ideas of 
dynamic programming [5] to a limiting case of a multi-move discrete 
game. A brief sketch of the history of the later mathematical develop- 
ment of differential games up through 1965 is contained in [22]. Isaacs 
published a major work in 1965 [30]. Y. C. Ho [23] has reviewed Isaacs's 
work and discussed its relationship to deterministic optimal control theory 
in an excellent review of the book Differential Games . 

The subject referred to as differential games may in the future 
be called zero-sum deterministic differential games within the emerging 
framework of "generalized control theory" [24], [25]. It seems appro- 
priate therefore to review briefly the characteristics of such 
optimization models. We consider two-controllers who manipulate their 
own control variables in a dynamic system whose behavior is described 
by a system of differential equations. Each controller has his own 



criterion function, but these are related by summing to a constant. 

Hence, one man's loss is the other's gain. There is one information 
set and it is perfect in the sense that all past history is known, 
opponent's capability, etc., except the instantaneous strategy of the 
opponent. It is within the framework of these general assumptions 
that idealized surveillance-evasion tactics will be developed. 

The work of Isaacs has been the major source of ideas for the 
current research. Although recent work has been more mathematically 
precise [6], [7], the worked examples in Isaacs book appear to this 

researcher to be at least a decade ahead of the development of those 
who place a premium on precision. Each new application of differential 
game theory appears to motivate several new concepts. 

b . Generalized Control Theory 

It seems appropriate to discuss the general problem of pursuit 
and evasion within the broader framework of "generalized control theory." 
As noted above, two notable deficiencies of the differential game models 
to be considered in this work are: (1) perfect information is assumed 

and (2) the model is deterministic. Hence, we will address only the 
kinematic aspects of surveillance-evasion and will not consider deception 
tactics . 

Within the past several years a probabilistic control theory 
has emerged. W. Fleming [19] recently has reviewed this field and 
provided an extensive (and basic) bibliography. Willman [50] and Behn 
and Ho [4] have extended these concepts to conflicting dynamical systems. 
Such an approach applied to the problem at hand would consider detection 
probabilities and that the players have only imperfect knowledge of the 
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state of the system, i.e., there is M noise" superimposed on the signal 
as to the location of one’s opponent. Such extensions are beyond the 
scope of the current modelling effort, but are noted for possible future 
extensions. The deterministic model is complementary to the stochastic 
model and should provide insight into the latter. 

c. Application to Problems of Naval Warfare 

We have seen that differential games provide a model for optimizing 
conflicting dynamical systems over a period of time. There are numerous 
applications of such models to problems of Naval warfare: 

(1) interception of enemy missiles by ABM's, 

(2) allocation of Naval fire support to various targets, 

(3) allocation of Naval airpower to ground-support and 
strategic targets , 

(4) allocation of effort in searching for targets, 

(5) surveillance and tracking of hostile vehicles. 

These various applications are noted, since the solutions of all these 
problems involve the use of the same mathematical technique, differen- 
tial game theory. 

In the current research, we study the problem of surveillance- 
evasion, in which the "pursuer" attempts to maintain contact with an 
"evader" who attempts to break contact by moving outside the detection 
capability of the pursuer. The mathematical structure of this problem 
is closely related to that of pursuit and evasion problems: in sur- 

veillance-evasion the problem occurs, for example, within a circle and 
terminates on its boundary, while in pursuit and evasion the problem 
is exterior to a circle. This research has uncovered some subtle differ- 



ences, however. 
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The discrete models of multi-move discrete games [5] (generaliza- 
tion of dynamic programming) may be used to study these phenomena under 
less restrictive assumptions. The value of the differential game 
approach is that it leads to explicit expressions relating the various 
system parameters. Thus, the basic structure of optimal policies and 
tradeoffs between system parameters may be explicitly exhibited. 

The mathematical techniques of control theory show great promise 
for providing insight into optimizing the dynamics of Naval warfare. 
Many previous analyses involving classical variational methods may be 
more easily done and extended by their use. As an example of this, 
de Guenin ! s extension [13] of Koopman's results [38] on the optimum 
distribution of searching effort is derived in Appendix E by use of the 
Pontryagin maximum principle. 



6 



II. REVIEW OF PERTINENT LITERATURE 



The published literature was reviewed to find out what had been 
done on the topic of surveillance-evasion in order to avoid duplica- 
tion of research effort. We do not attempt a comprehensive review of 
the literature, since that was not the purpose of this research. How- 
ever, some major works are highlighted . Literature was reviewed in two 
subject areas: search theory and differential games. 

J. Dobbie [15] has published a rather comprehensive survey of 
search theory in 1968. He indicated that the only published work on 
the tracking operation in the open literature was by Dobbie himself 
[14]. The 1966 paper by Dobbie considers the kinematic aspects of 
surveillance-evasion. This paper is the primary basis for the present 
research. Dobbie considers a sequence of problems, formulated as differ- 
ential games: surveillance-evasion for a circular detection region of 
the pursuer, tracking for an arbitrary detection region, and two models 
in which recontact is possible by the pursuer. The content of this paper 
has evolved into operational Navy doctrine [16]. 

A more extensive search of the literature did not yield any further work 
on tracking operations in the open literature. The 1966 survey by 
Enslow [18] and S. Pollock* s selected bibliography [42] were consulted 
in this respect. Both these surveys were consulted by Dobbie for his 
survey article [15]. A recent effort at the University of Michigan [41] 
was also examined and did not yield any new references on the tracking 
problem. 

The differential game literature was consulted for general mathe- 
matical background and to see if applications to tracking could be found. 
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The applications literature was considered of prime importance rather 
than the development of the mathematical theory. Isaacs ! s 1965 book 
[30] remains the chief source of examples and insight into technical 
questions related to solving actual problems. His terminology is non- 
standard to control theorists, little use or reference to the classical 
variational methods is made, but he does provide an extensive theory 
naturally motivated through examples. Isaacs homicidal chauffer problem 
is basic to pursuit and evasion studies. He had also considered 
the tracking problem while at CNA [31]. This reference does not contain 
any analysis and the condition developed for surveillance to be main- 
tained is incorrect (as is that developed by Dobbie [14]). Isaacs* s 
sketches of the problem indicate that he did note, however, the termina- 
tion of the barrier. 

L. Berkovitz*s paper [6] presents an extremely rigorous mathe- 
matical development of differential game theory but presents no examples 
and does not consider most of the significant aspects required to solve 
specific problems. Blaquiere, Gerard, and Leitman [8] have recently 
published a book on differential games. This book develops the theory 
from a geometric point of view and is an extension of the geometric 
approach to control theory problems developed by these authors over the 
past five years. This previous work is accessable through the biblio- 
graphy in this book. Although some examples are given, they don*t 
appear to be representative of a broad experience in applications as 
is the case of those in Isaacs* book. This book is not written for one 
not already acquainted with the theory and is not useful for the novice. 
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The intimate connection between differential games and control 
theory is pointed out in numerous places (see, for example [6], [23], 
[30]). In view of this relationship, much of the optimal control theory 
can be brought to bear on differential game problems. An excellent 
review of deterministic optimal control theory with an extensive 
bibliography is by Athans [1]. An excellent, concise discussion of the 
relationships between control theory and differential games is contained 
in the review of Isaacs book by Y. C. Ho [23]. Other articles which 
contain useful review material are by Ho, Bryson, and Baron [22] and 
also Sarma and Ragade [45]. Athans and Falb [2] have written an excellent 
introductory text. The excellent book by Bryson and Ho [9], besides 
being an easily understood, lucid introduction to the field, contains 
many advanced topics including a brief introduction to differential 
games . 

The Russians have done extensive research on optimal control/ 
differential games over the last decade [3]. An excellent survey article 
delineating numerous fields of application and with an extensive biblio- 
graphy of original Russian research papers is by Simakova [46]. Y. C. 

Ho, one of the best qualified individuals to survey the current Western 
state-of-the-art, has documented current developments (theoretical and 
in applications) [25]. An important question in pursuit and evasion 
problems is "When can capture occur?" Isaacs [30] has developed several 
(equivalent) criteria for determining the useable part of the terminal 
surface. Recently, L. Meier [40] has proposed a new geometrical criterion 
from his study of ABM interception of re-entry vehicles. 
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III . The Surveillance-Evasion Problem 

We consider an idealized model for the problem of a tracker 
(for example, a destroyer) keeping a hostile vehicle (for example, 

a submarine) under constant surveillance. The maneuverabilities (i.e., 
maximum speed and turning capabilities) of the two vehicles lead to the 

appropriate conditions. When these conditions are met, the pursuer can 

* 

keep the Evader under close surveillance. The Evader cannot do anything 
to break contact and prevent this ’’tracking" or "tailing." We consider 
an extremely idealized model with perfect information on the location 
of the enemy for both antagonists and a circular, "cookie-cutter" 
detection region for the Pursuer. 

a. Statement of the Problem . 

We consider the same model used by Dobbie [14] , which is an exten- 
sion of Isaacs’s model [30]. There are several flaws in the earlier 
mathematical development of Dobbie, which lead to an incorrect condition 
for surveillance to be maintained and an incorrect analysis for surveil- 
lance-evasion with arbitrary detection regions. The latter has the 
important implication that the involute tactic [16] may not be optimal 
for holding contact in real world situations where sonar capabilities 
generate a non-circular detection region. It should also be noted that 
the brief work by Isaacs on this problem also yielded the wrong surveil- 
lance condition. Hence, the purpose of the present research is to set 
such analysis on a firmer mathematical basis. 

In the model the Pursuer is faster than the Evader, who does, 
however, possess an advantage in turning capability. We assume a 
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circular "cookie-cut ter M detection region for the Pursuer, who detects 
the Evader with probability one when they are less than a distance d 
apart. The goal of the Evader is to break contact as quickly as possible 
by moving out of the circular detection region of the Pursuer, who 
attempts to maintain contact as long as possible. We define the follow- 
ing notation : 

subscripts: 1 refers to Pursuer, 2 refers to Evader 

s^ = Pursuer’s speed with maximum w^ 
s^ = Evader’s speed with maximum w^ 

R = minimum turning radius of the Pursuer 

<p = fraction of maximum course curvature employed by Pursuer 

(4> = -1 corresponds to left turn with minimum turning radius) 

ip = Evader’s heading relative to that of Pursuer 

d = radius of Pursuer’s detection region 

T = time for Evader to escape (reach circle of radius d from 
Pursuer) 

Thus, the problem facing the Pursuer is 

T 

max min [ dt, 

0 

subject to the equations of motion of the two vehicles. It is conven- 
ient to adopt a relative coordinate system for two reasons: (1) the 

dimension of the problem is reduced and (2) such a coordinate system 
is standard in Naval operations. In this relative system the Pursuer 
is located at the origin and has a verticle heading. Thus the coordinate 
system is ’’carried by the Pursuer." In this system the problem is 
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max min 
*’ s l ^ ,s 2 0 



dt with T unspecified. 



dx S l y * 

subject to: tt = - 1 s . sin tK 

J dt R 2 



djr V* , 

^ = — + s 2 cos * - s r 



and -1 £ <f> £ 1, 

0 £ s^ £ w^, 

0 ^ s 2 ^ w 9 < w^, 

with initial location of evader 



( 1 ) 



x (t = 0) = x Q , 
y (t = 0) = y Q , 

and terminal surface defined by 
x 2 (T) + y 2 (T) = d 2 . 

These equations are derived in Appendix A, since the solution procedure 
relies on an understanding of the geometry of this relative coordinate 
system. 

b . Games of Degree and Games of Kind . 

In this section we discuss the concepts of a game of degree and 
a game of kind. The purpose of this discussion is to explain that in 
order to solve a game of kind one must solve that part of a correspond- 
ing game of degree into which the game of kind has been imbedded. The 
common part of optimal strategies for these two games is the barrier, 
i.e. boundary of domain of controllability. 
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Isaacs ([30] p. 35) defines a game of degree as one with a contin- 
uous range of payoffs. In the surveillance-evasion game, the formulation 
of the previous section is a game of degree with time of escape as the 
payoff. A game of kind is one with a finite number of payoff values. The 
reaching of each particular terminal state yields a single payoff value. 
Such is the case if we consider the payoffs as 4-1 if contact is broken 
and -1 if the Evader can’t break contact. Blaquiere et al ([8] pp. 9-10) 
have a similar definition except that a quantitative game (game of degree) 
is a game in which there is a common target set for both pursuer and 
evader. It should be noted that there are attrition games in the liter- 
ature for which this definition is inadequate [47]. 

We define the domain of controllability of a terminal state to be 
that subset of the initial state space from which trajectories lead to 
this terminal state for all admissible strategies of the player for 
which this terminal state is unfavorable when the extremal strategy of 
his adversary is played. By an extremal strategy, we mean a strategy 
determined on an extremal trajectory, which is a path on which the 
necessary conditions for optimality [6] are almost everywhere satisfied. 

The barrier, being the boundary of the domain of controllability, is 
the trajectory which leads to the boundary of the useable part of the 
terminal surface. The useable part of the terminal surface is that part 
of the terminal surface to which there are paths from the state space 
(see Issacs [30] p. 83 and also Appendix B) . 

Thus Isaacs ([30] p. 13) imbeds a game of kind into a game of 
degree. We may consider a game of degree as a game of kind in which 
the prerogative of each of the antagonists is exercised to do his "best. 11 
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The point being developed is that o ne must know how to solve a 

game of degree in order to solve the corresponding game of kind . Other 
uses of the game of degree solution are 

(1) show path of system when prerogatives are exercised, 

(2) when the prerogative is not exercised by one party, indicates 
direction system will move. 

The latter remark is the motivation for Isaacs T s concept of a semi- 
permeable surface ([30] pp. 70-71). 

Let us discuss further the concept of a semi-permeable surface, 
since even one of the leading control theorists has overlooked the 
reason for this concept in his excellent review of Isaacs’s book [23]. 
Initially, we consider a game of degree and a simply-connected 

domain in the state space. When the problem has a solution, this 
domain is covered by a field of extremals [17]. We can transform our 
original problem to one with terminal payoff, so that the value of the 
game, denoted by V, will be constant along an extremal. Consider now 
Figure 1. Three extremals with the accompanying value of the game are 
shown. If extremal strategies are used by both players, the trajectory 
remains on the extremal. If the Pursuer uses his optimal strategy 4> , 
but the Evader uses a non-optimal strategy, then the course of the 
system is steered to lower payoff values and similarly when the roles 
are interchanged. 

To reiterate, the semi-permeable surface is a surface in the 
state space fcr which each player controls the penetration by the path 
of the system by use of his optimal strategy. When a player employs 
his optimal strategy, his opponent can’t steer the system to a more 
favorable position by any strategy, and if he doesn’t use his own optimal 
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(p - Pursuer (minimizes) 
ip - Evader (maximizes) 



Figure 1. Semi-permeable Surface. 
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strategy, a less favorable outcome will result. This situation is a 
consequence of the criterion functional having a saddle point when 
optimal strategies are used. The purpose of Isaacs 1 introduction of this 
concept of a semi-permeable surface is that the barrier, the only place 
where optimal strategies are determined for a game of kind, is a semi- 
permeable surface. 

The barrier is a surface in the state space (Isaacs refers to the 
state space as the "playing space”) which divides the captur e and escape 
zones when both exist. The global answer to the capture-or-escape ques- 
tion depends on whether or not the barrier divides the state space into 
two parts. Hence, we know that if we can show that the barrier termi- 
nates without having done this, the entire state space is either all 
capture zone or all escape zone. For the problem at hand (surveillance- 
evasion) this means that there is probably no surveillance zone if the 
barrier so terminates. Termination of the barrier is discussed in 
Isaacs 1 s book [30] on pages 210-214. The argument given there that a 
barrier terminates due to an abrupt change in direction (see Figure 2) 
is best understood by recalling that a game of kind is imbedded into a 
game of degree. In this game of degree, each behavior would lead to 
more than one extremal at a point in the state space. 

c . Connection with the Homicidal Chauffer Game 

The research philosophy has been to employ a "broad" approach to 
specific problems by drawing upon theory from diverse fields. For 
example, consideration of the geometric properties of complex numbers 
has led to a geometric way to construct extremal paths in the homicidal 
chauffer game. This led to the correct condition for termination of the 
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k. Barrier 






Figure 2. Conditions for Termination of Barrier. 
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barrier (and hence the existence of no surveillance region) , which had 
eluded both Dobbie [14] and Isaacs [31] • 

Consideration of a sequence of closely related problems has been 
attempted to try to learn from their common points. Three such 
closely related problems are: 

(1) destroyer to fixed destination, 

(2) homicidal-chauffer game, 

(3) surveillance-evasion game. 

In working on the surveillance-evasion game, consideration of the first 
two problems has proven to be useful. The study of these has many 
points in common with the surveillance-evasion game. Hence, analysis 
of these problems is presented in Appendices B and C as background 
material. 

Specifically, the relationship of the homicidal-chauffer game and 
the surveillance-evasion game is as follows (as first pointed out by 
Isaacs [31]). Consider the terminal surface of the homicidal-chauffer 
game. It is a circle, and the state space for the game is exterior to 
this circle. For the surveillance-evasion game, the state space is 
interior to the terminal surface with the useable and non-useable parts 
of the terminal surface being interchanged. A point worth noting (again 
first pointed out by Isaacs [31]) is that most pursuit differential games 
can be converted into surveillance games by turning the analysis inside 
out . 

d . Solution of the Surveillance-Evasion Game 

In this section, we show the solution to the game of degree. Analysis 
details are presented in Appendix D. As noted in Section Illb, we may 
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consider the game of kind as imbedded in a game of degree. Hence, when 
we consider the problem as a game of kind (Is it possible for surveil- 
lance to be maintained?) , optimal strategies of the game of degree only 
apply on the barrier for the game of kind. 

Before discussing briefly the geometric aspects of solution, let us 
summarize the new results of the current research: 

(1) correct condition for surveillance to be maintained, 

(2) new geometric construction for escape paths, 

(3) extension of model to non-circular detection region. 

An important question to be answered by the model is, "Under what 
circumstances can surveillance be maintained?" The correct answer is 
when 

d ^ R{/l -(w 2 /w 1 ) 2 + 2 (w^/w^) (tt - U) }, (2) 

where 



cos li = an< ^ 0 <> U ^ tt/ 2 . 

Dobbie [14] and Isaacs [31] had derived different conditions. 

We also have discovered a geometric interpretation for the optimal 
escape paths. For 0 £ t £ 2R/w^(tt - u) , the path equation may be 
written as 



x(x)- R 




cos w^ t/R 


sin w^ t/R 


(d - t v^sin u - R 


. y(r) , 




-sin w t/R 


cos w t/R^ 


, (d - T W )coS U / 



where U £ u ^ tt / 2 and cos U = W 2^ W 1* 0 ^ U £ tt/ 2 . 

t = T - t i.e., time measured backwards from escape. 
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This equation says that the Evader's location on an optimal escape path 
may be obtained as follows (with reference to Figure 3) : 

(1) locate escape point on detection circle, for example A^, 

(2) point moves along line OA^ with speed w^ towards 0; 
at time x it is at A^ , 

(3) rotate A^ through angle w x/R in negative sense about 
point (R,0); this yields , the point on the optimal 
trajectory at time x, 

(4) maximum possible rotation is through angle 2 (tt - u) , where 
u is angle between line from escape point on detection 
circle to 0 and the positive y-axis. 

At a later time x^, the point has moved to A^ and is rotated to 
A^ on the escape path. When the optimal trajectory is the barrier, 
there is an additional special geometric property of the escape path. 

It may be shown that the barrier is an involute to a circle of radius 
R ^ 2^1 with center (R,0). Hence, at each point on this curve, the 
normal is tangent [12] to the circle just mentioned. This is illustrated 
for point A^ in Figure 3. For non-barrier escape paths (consider path 
BiB^B^) , the curve is not an involute to any circle, but the same geo- 
metric construction holds. 

This geometric construction provides deeper insight into the 
geometry of escape paths. (A similar geometric construction is possible 
for both the destroyer to fixed destination problem and the homicidal 
chauffer game.) It suggests that there may be a conjugate point [44] 
to the escape point on an optimal trajectory, i.e., neighboring extremals 
intersect at this point. Such a point is B^ in Figure 3. Further 
investigation of this phenomena seems warranted but time hasn ! t permitted 
it. The geometric construction suggests that the problem may be more 
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Figure 3. 



Geometric Construction for Optimal Escape Paths, 
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easily solved in relative polar coordinates, but this has not been 
explored too far as yet. A similar construction results when the solu- 
tion is extended past the time restriction above. We note that the 
y-axis is a line of symmetry for this problem. 

We have briefly examined this problem for a non-circular detection 
region. Our research indicates that Dobbie f s [14] approach is incorrect 
and that the problem should be re-examined for some simple non-circular 
geometries. This implies that the involute tactic [16] may not be 
optimal for non-circular detection regions. The justification for 
these statements is given in the next section. 

We now consider the geometry of the solution to the tracking pro- 
blem. We shall describe the optimal trajectories (as far as the current 
research has progressed) and optimal tactics. The type of geometric 
configuration for the escape paths depends on the craft speeds, the 
Pursuer T s minimum turning radius, and the radius, d, of his detection 
region. We let 



d^ = R{/l “(w 2 /w 1 ) z + 2 (w^/w^Ctt -U)}, 



(4) 



d 2 = R{ /l -(w^/w^ 2 + (w 2 /w 1 ) ( I* 7T - U ) + 1} , 



(5) 



where 



cos ll = w 2 /w^ and 0 £ U £ tt/2. 



Then, there are three cases (see Appendix D for details) : 



(1) d < d 1 

(2) d 1 £ d < d 2 



no surveillance region (of type in next two cases) 



(3) d 2 s d 



barrier meets the negative y-axis. 
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These three cases are shown in Figures 4 through 6. In all cases the 
barrier is the involute to a circle of radius R w^/w^ and center 
(+ R,0). The tangents to these circles from 0 are of special signi- 
ficance. The escape paths are symmetric with respect to the y-axis, 
and hence we discuss the solution only in the right half-plane. Let 
u be the angle between line from escape point on detection circle to 
0 and the positive y-axis. The equation of escape paths is given 
by (3) for range of x given there. 

In case (1) , the barrier terminates when it reaches the circle 
of which it is the involute (see AA’ in Figure 4) . Paths which terminate 
between A and B are given by equation (3), but it hasn’t been ascer- 
tained whether such paths terminate abruptly (yielding a dispersal surface) 
for x £ x^ = 2 R/w^(tt - u) due to intersection of neighboring extremals. 
Escape paths have not been traced backwards for x ^ x^. Thus, although 
there is no surveillance region of the type for cases (2) and (3) , the 
optimal escape paths have not been determined from all of the state space, 
and there may be "surveillance pockets" present. Hence, we have estab- 
lished the Evader’s escape tactics only in the small region AA’CB, when 
he is close to the limits of the Pursuer’s detection capability. One 
disturbing feature of this model is that for escape at x(T) = d sin u 
and tt/2 < u < 3/2 tt , the Pursuer’s optimal tactic is to stop dead in 
the water, s^ = 0. Such escape paths (shown in Figure 4) terminate 
at D, E, and F and originate from 0. 

In case (2) , the barrier divides the state space into a surveillance 
zone and an escape zone. For 0 £ x £ x^ = 2 R/w^(tt - U) , the Pursuer 
uses <J> = 1 (sharpest turn to right). For x > x , the Pursuer uses 
4> — —1 , and the barrier is given by 
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Figure 4. Geometry of Tracking Problem, d < d^- 
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x(t)+R 




cos w^(t-t^)/R -sin w^(t-t^)/R 


x(t^)+W2 (t-t^)cos (tt/2- U)+R 


y(-r) / 




sin w^(t-t^)/R cos w^(t-t^)/R 


.y(T- L )-w 2 (T-T 1 )sin(ir/2-U) , 



which may also be shown to be an involute unwinding from a circle with 
center at (-R,0) and radius R At T = T ]_> th e P°i nt D is 

always located on the lower tangent to the circle as shown in Figure 5. 

It lies between 0 and D 1 (when d = d^, it is at D*). The paths 

terminating between A and B intersect each other, but complete 

details have not been worked out as yet. Again, for escape at x(T) = d sin u 

and tt/ 2 < u < 3/2 tt , the escape paths are straight lines terminating 

at, for example, F, G, H, and I. Paths in the vicinity of ODE (a "pocket") 

haven* t been worked out. 

In case (3), the Pursuer only uses <|> = 1 on the barrier, since 
it intersects the negative y-axis before x = x^. When d = d^, the 
barrier is tangent to the negative y-axis. Other aspects are similar 
to above. This case is shown in Figure 6. 

e . Shortcomings of Previous Work 

The differential game solution techniques of the current research 
differ from Dobbie*s approach [14]. It is the purpose here to discuss 

such differences, since some of our results differ from his. Dobbie, 

considering a game of kind, uses Isaacs’s "game of kind approach," 

(see chapter 8 in [30]). As we have discussed above, Isaacs developed 

this approach by imbedding such a problem in a game of degree. The 

analytic details of many steps in the solution of a game of kind (see 

[30] pp. 205-210) will be seen to be the same as employed here (game 

of degree) with the vector of dual variables p replaced by the normal 

to the barrier v- The Hamiltonian is also modified slightly. However, 
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Figure 5. Geometry of Tracking Problem, £ d < d^. 
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Figure 6. Geometry of Tracking Problem, £ d. 
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two important aspects are not adequately treated in Dobbie’s work [14]: 

(1) termination of the barrier (see [30] pp. 210-214), 

(2) construction of the barrier (see [30] pp . 214-215). 

As we have discussed above in section Illb (see also Appendix B) , 
whether or not there is a surveillance region is dependent upon whether the barrier 
terminates or not. This aspect of solution is ignored by Dobbie, whose 
condition for the existence of a surveillance zone should be contrasted 
with ours. 

A more serious criticism must be leveled at Dobbie 1 s method of 
determining the useable part of the terminal surface, denoted as UP. 

Dobbie does not make use of the fact that every game of kind is imbedded 
in a game of degree. Hence, he does not recognize that the solution 
depends on the geometry of the detection region and erroneously concludes 
that the solution for a circular detection region would apply for an 
arbitrary detection region ([14] p. 177). The purpose of criticism 
of Dobbie 1 s results is to point out that the surveillance-evasion problem 
has not been solved for arbitrary detection regions and suggest such a 
task as a future research effort. 

Let us discuss why the solution depends on the geometry of the 
detection region. Isaacs ([30] p. 215) states a criterion for the 
construction of the barrier: the normal to the barrier coincides with 

the normal to the terminal surface. This leads to our major criticism 
of this pioneering effort: Dobbie tried to extend the model's solution 

for a circular detection region to arbitrary detection regions when 
such an extension is not justified. 
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We consider a second argument (from optimal control theory) for 

solution dependence on geometry of detection region. Dobbie (p. 174) 

says, "Let t = 0 at a point A of a barrier for which yv^ = xv^ , 

so that the normal lies along the radial line OA." Since t = 0 is 

time of escape, the barrier is tangent to the detection region at 

escape (normal of barrier is perpendicular to escape surface) for a 

circular detection region but does not have to be for an arbitrary 

detection region. It is well-known in control theory (see [2] p. 290) 

-y — y ~y 

that p is parallel to n where p is vector of dual variables at 
terminal surface and n is normal to terminal surface (pointing inward 
to state space). For the problem at hand, the normal to the barrier 
(in Dobbie f s notation) is parallel to p. Hence, this normal must be 
perpendicular to the detection region at the moment of escape, and this 
condition may be violated in Dobbie f s analysis for an arbitrary detection 



region. 
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IV* Conclusions and Future Extensions 



Here we summarize what we have done and suggest possible future 
research* We think that we have established more firmly the mathematical 
basis of a certain type of surveillance-evasion model. Specifically, 
we have accomplished the following: 

(1) parts of the surveillance-evasion game of degree have been 
solved (A disturbing aspect is that the Pursuer stops still 
in the water for those cases when the Evader escapes "behind 
him. ") . 

(2) correct condition developed for surveillance zone to exist, 

(3) devised geometric construction for describing optimal escape 
paths , 

(4) showed that Dobbie's extension of the solution to arbitrary 
detection regions was incorrect. 

Based on this research effort we suggest the following as possible future 
work : 

(1) develop further the solution to the game of degree (This 
would provide insight into Evader escape paths and tactics, 
especially for those cases when contact can be broken, i.e. , 
no surveillance zone.), 

(2) examine problem for non-circular detection regions (This 
would allow actual sonar patterns to be more accurately 
described in the model. We suggest that analysis first 
consider an elliptical detection region and then try to 
generalize results . ) , 

(3) consider surveillance-evasion game in relative polar 
co-ordinates (This approach is suggested from new geometrical 
construction noted above and is related to tasks (1) and 

(2) above . ) , 

(4) study extensions of basic surveillance-evasion scenario, 

(a) formulate problem which eliminates a stationary Pursuer 
as an optimal tactic, 

(b) study effects of Evader having maneuverability limita- 
tions (What quantitative effect does this have on 
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condition for Pursuer to have surveillance zone? This 
would be an application of the extension of Isaac f s 
homicidal chauffer game called the game of two cars 
[30] p. 237.), 

(c) study problem of two Pursuers against a single Evader, 

(d) develop other models of tactical interest and study 
other extensions in the literature. 



31 



APPENDIX A. DERIVATION OF BASIC EQUATIONS. 

In this appendix we derive equations (1) of the main text. Even 
though these equations are briefly derived on p. 30 of [30], we feel 
that the current derivation has increased our understanding of the 
relative coordinate system and may be useful to others. First we 
translate the restriction of a finite, non-zero minimum turning radius 
into a restriction of the maximum rate of change of direction. Next, 
we develop the basic equations in a fixed reference frame. Finally, 
the equations are transformed to the relative coordinate system. 

a. Implication of Finite, Non-zero Minimum Turning Radius . 

We consider motion of a point in the plane when the radius of 
curvature is bounded below and greater than zero. We assume that the 
point is moving with constant speed w and adopt a fixed rectangular 
frame of reference as shown below. The curve is given parametrically 
by 




>- x 
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x = x(t) and y = y(t), where t is time. Hence, the velocity 
components are given by 



dx 

— = w cos 0 , 



iLZ. - 



dt 



= w sin 0, 



(Al) 



If there were not restrictions on maneuverability, then we could choose 
0 = 0 (t) in a completely arbitrary manner. Let us now see how a lower 
bound on the turning radius restricts 0 = 0(t) through -^7. 

We consider a curve in the plane given by x = x(s) and y = y(s), 
where the parameter, s, is arc length. The curvature , k, is defined 
as the rate of change of angle of inclination, 0, with respect to arc 
length (see pp . 280-282 in [12]) 



K 



lim 

As-K) 



A 0 

As 



d0 

ds 



where the angle of inclination is shown in the figure below and the 

dy 

slope of the curve is related to 0 by tan 0 = 




33 



The radius of curvature of the curve at a point is defined by p = 1/ K . 
The restriction that the radius of curvature is bound below may be 
expressed as p ^ R > 0 or k £ 1/R. Introducing a new "turning” 
variable, we may write this as an equality 

k = <f>/R where -1 £ $ <> 1. (A2) 



Since time is a more convenient parameter than arc length, we have 



d9 _ d_0 eft _ d9 ,ds 
K “ ds “ dt ' ds dt at ’ 

Recalling that ds = /ds^ + dy z , we have that 

v< 



ds 

dt 




(A3) 



(A4) 



by use of (Al) . Combining (A2) , (A3), and (A4) , we obtain 



d9 

dt 



(f)w/R where -1 £ <f> <> 1, 



which is the desired restriction on 0(t) from the lower bound on 
the "turning" radius. We summarize the equations of motion in a plane 
when there is a lower bound on the radius of curvature, R. 



dx 

dt 



= w cos 



0 , 



& = 
dt 

d0 

dt 



w sin 9 , 
<j>w/R where 



-1 £ <f> £ 1. 



(A5) 



b. The Relative Coordinate System . 

It is convenient to adopt a relative coordinate system , one that 
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moves with the observer. Our development is the same as Koopman's 

([38] or again [39]). We consider first the simple case of motion at 

fixed speed and course for an observer (who carries the coordinate 

system with him) and a target. Assuming both the observer and the 

target move at constant speeds in straight lines, we let 

v = speed of observer in knots (ocean or true speed) , 

u = speed of target in knots (ocean or true speed) , 

w = speed of target relative to observer in knots. 

The corresponding vectors, v, u, and w, describe the constant 

-¥• 

motion. We note that the vector difference, w = u - v, gives the 

motion of the target relative to that of the observer. Figure A1 shows 

the relationship between true and relative velocities and angles. In 

Figure A1 the vectors are "laid off" from the same point. The actual 

situation is shown in Figure A2 . 

Thus, we consider a coordinate system moving with the observer 
(see Figure A3) . 

(or Pursuer). 1 The y-axis of this coordinate system is coincident with 
the observer 's velocity vectory. P is a fixed point, located at the 
origin. The point E (which may be either moving or stationary in 
the fixed reference frame) moves relative to P. We later derive the 
equations of motion of E relative to P. We note that in this rela- 
tive coordinate system, there are two factors contributing to E f s 
motion : 

(1) rectilinear motion of P and E, 

(2) rotation of coordinate system when P turns. 



(o bserver v eloc ity) 
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Figure Al. 



True and Relative Velocities and Angles. 
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Figure A2. 



Target's Track Relative to Observer. 
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W- 




ip is relative heading of Evader 



x 



Figure A3. The Relative Coordinate System. 



c. The Basic Equations in a Fixed Reference Frame . 

We consider the motions of a Pursuer, P, and Evader, E, in 
a stationary coordinate system. We consider the problem of the Evader 
trying to break contact with a Pursuer by moving out of the Pursuer 1 s 
detection region, which is circular with radius d. We use the 
notation defined in section Ilia, of the main text. The situation is 
shown below. 

y f 
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We have seen above that the non-zero minimum turning radius, R, of 
the Pursuer yields a restriction on the rate of change of a 



da 

— = <J>s_./R where -1 £ <j> £ 1 
d t i 



Hence, in the stationary coordinate system the surveillance-evasion 
problem may be stated as 



max min | dt with T unspecified, 
’ S 1 B ’ s 2 0 



subject to: 



dx i 

~~z — = s., cos a , 
at 1 

dy i 

dT ' s i Sln “• 



4^- = 4> s i /R where -1 £ 6 £ 1, 
dt 1 

dx 2 

IT = s 2 cos 6 ’ 

dy 2 

dF = s 2 sin 6 ’ 



where 



0 £ s^ <: w^ 



0 £ s^ £ w^ 



with initial conditions 



x x (t=0) = x° , y 1 (t=0) = y° , a(t=0) = a Q , 



x 2 (t=0) = x° , y 2 (t=0) = y°, 



and terminal condition 



[x 1 (T)- x 2 (T)] 2 + [y^T)- y 2 (T)] 2 = d 2 
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d. The Basic Equations in the Relative Coordinate System . 

We now transform the above equations to the relative motion 
coordinate system discussed previously. In this new system x is the 
distance the Evader, E, is from P in a direction measured perpendi- 
cular to P f s heading. In this relative coordinate system there are 
two factors leading to the apparent motion of E: 

(1) rectilinear motion of P and E, and 

(2) rotation of the coordinate system when P turns. 

The components of velocity due only to translation are given by (see 
Figure A3) 



since P ! s motion is always directed along the y-axis. We next derive 
the components of velocity due only to rotation with angular velocity 

dQ 

a) = — . When P turns to his right (<(> = 1) , E is rotated counter- 
dt 

clockwise about P. We take u) to be positive when counterclockwise. 




s 2 sin ip , 



t 




Then 




r 




r 



Using the fact that 



dx 

dt 




r 



equations (1) of the main text are readily obtained. 
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APPENDIX B. DESTROYER TO FIXED DESTINATION 

In this appendix we derive parts of the solution for the best way to 
steer a vehicle with a minimum turning radius to a fixed destination in 
least time. We call this problem "destroyer to fixed destination." It 
is of special significance, since it is the limiting case of the homicidal 
chauffeur game when the Evader's speed goes to zero. Study of this problem 
has increased our insight into these pursuit-evasion problems. The new 
geometrical construction for optimal paths was first suggested in our 
study of this problem. As a general principle, many times most of the 
significant solution aspects of a differential game may be studied by con- 
sidering a one-sided version of the problem. 

We state the problem and then present the details of analysis. Next, 
we discuss our new geometrical construction for optimal trajectories in 
such problems. Finally, we discuss the geometry of the solution and 
summarize the analysis results. Many solution steps and aspects are ex- 
plained in elaborate detail in this appendix. 

a . Statement of the Problem . 

The problem is to determine how to steer a constant speed vehicle 
with minimum turning radius, R , from any point, (x^,y^) , in the plane 
to within a distance, l , of a terminal point, 5 -^ east • 

In a stationary coordinate system the problem is 

T 



min 

(f) 



o 



dt with T unspecified, 
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subject to: 


dx 

= w cos 0 , 

dy . 

~r~ = w sin 0 , 
at 




= (j)w/R where -1 < <b < 1 , 
dt — — 



with initial conditions 

x(t = 0) = x 1 , y (t = 0) = y 1 , 6(t = 0) = , 

and terminal condition 

[x ( t = T) - x 2 ] 2 + [y (t = T) - y 2 ] 2 = £ 2 

We transform this problem to the relative coordinate system of Appendix 
A in the same fashion as shown there to obtain: 



subject to: 


f T 

min dt with T unspecified , 

<J> ^ o 

dx w 

dt R y<t> ’ 




^ x4> - w where -1 < <j> < 1 , (Bl) 

d t R 



with initial conditions 

x (t = 0) = x Q , y(t = 0) = y 0 , 

and terminal condition 



x 2 (T) + y 2 (T) = £ 2 



b . Development of Solution 
Hamiltonian, H(t, x, p, <p) 
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h ( t , x, p, <f>) = l + y4>) + p 2 (|t x< t> -w) , 

where p.. = t—— and p 0 = — — are dual variables and J* 
i dX La y 



/W 



(B2) 

rT 

min dt . 

(J) ' o 



We determine the extremal control from 

min H (t , x, p, <}>) min{^ <f>(p 0 x - p n y} subject to -1 £ <|> £ 1. 

i . K- Z _L 

4 > 4 > 

Hence 

4> = sgn(p 1 y - p 2 x) = sgn A(t) , (B3) 

f+1 for x > 0 

where A(t) = p..y - p^x and sgn x = \ 

>1 for x < 0 

s 

Boundary Conditions for Dual Variables 

Since termination is any point on a curve, we must have p = 
normal to this curve at t = T(terminal surface) (see [2] p. 290). We 
let n be unit normal to circle (terminal surface) pointing into the 

— >■ —V -> —>■ 

state space. Then n = sin sit cos s j , and we have p = an . Since 
the constant is arbitrary, we let it be 1 and hence 

p^(t = T) = sin s , P2^ t = T) = cos s . (B4) 



Usable Part of Terminal Surface 

Because of the nature of the relative coordinate system, it is not 

possible for paths from the state space to end anywhere on the terminal 

2 2 2 

surface x (T) + y (T) = l . Physically, the terminal surface is a circle 
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about a moving point. In a fixed coordinate system this circle moves 
with the point. Consider a line perpendicular to the point's velocity 
vector and passing through the point. This line divides the circle into 
two parts: one part is ahead of the point’s motion and the other is be- 

hind. The problem ends when a fixed point, the destination, crosses the 
moving circle. Clearly, forward motion of the point can never cause a 
fixed point to cross the rear half of the circle. In the relative coordi- 
nate system, we can describe this condition mathematically as n • X < 0 , 
where n is unit vector normal to the terminal surface and pointing into 
the state space and 

dx ' 

-> dt 

X = 

dx 

dt 

is the vector of velocity components in the relative coordinate system of 
the moving point (destroyer) . That part of the terminal surface for which 
’’capture” can occur is called the useable part (see p. 83 of [30]). 

For the problem at hand we have 

n = sin s i + cos s j , 

X = (-£ y<j))i + X<J> - w)j , 

where the terminal surface is parametrically represented by s as shown 



below. 
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y 



A 



/ 



-> 

n 



> s 



1 



x' 



*- 



l 



Hence, we have 

x(t = T) = £ sin s and y(t = T) = l cos s. 

Then 

X (t = T) = cj) l cos s)i + (~ <j) l sin s - w)j , 

R R 



and hence 

n • X(t = T) = (- — ()) l cos s)sin s + (—<() i sin s - w)cos s 

R R 

or 

n • X(t = T) = -w cos s <_ 0 , (B5) 

defines the usable part of the terminal surface. Thus cos s >_ 0 for 
capture to occur and the useable part, UP , is given parametrically by 

UP = { s | - ~ £ s <_ y} • (B6) 

Equation (B6) simply says that "capture" only occurs in the "front half" 
of the circle about the moving point. 

The boundary of the use&le part of the terminal surface, BUP , divides 
the terminal surface into ussable and non-usedsle portions (see p. 83 [30]). 
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The paths which terminate at the BUP are then the barrier, for they 
separate the state space into regions from which paths either do or do 
not n lead directly" to the UP. We have noted in the main text that Isaacs 1 
concept of the barrier corresponds to that of the boundary of the domain of 
controllability in the control theory literature ([23]). We now see that 
we can determine the barrier by projecting the BUP backwards in time before 
capture. 

The Adjoint Equations and Their Solution 
We have that 

dp l 9H w 

dt 9x P 2R* ’ 



dp 2 w 
dt 9y P 1 R 

Since the boundary conditions for the dual variables are given for t = T , 
it is convenient to let t = T - t and integrate the adjoint system back- 
wards from the end. Accordingly, we obtain 



dp 

r l w 

dT P 2 R 



p^ (x = 0) = sin s , 



dp 0 

2 _ w 

dt ^1 R 



^2^ T = = cos s 



We may combine these equations as follows: 



4, 



dj 4 



^ P 2 w , ,w2>.2 ^ P 1 ,w$>, 2 

“dt R * ’ “ P 1 ( T ) ° r ~~1 ( T> P 1 

di 



0 
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Hence 



p (t) = A cos ^r- t + B sin ^ <J>t 
IKK 

with initial conditions 

P^(t = 0) = A = sin s 



dPi 

di 



(t = 0) = ^ 4>B = p 2 (t 



A\ w j. w , „ 

= 0) — (j) = — <j> cos s •> B 



COS s . 



Substituting for the constants and simplifying we find 



P^(t) = sin(s + ^ 4 >t ) . 

Similarly, it may be shown that 

P 2 (t) = CO s ( s + ^ <f>l) . 

Solution to the State Equations 

In the "backwards time" we have 

w . 

= — (py x (t = 0) = £ sin s , 

K 

= - (f)X + w Y(x = 0) = £cos s . 

K 



dx 

di 



dy 

di 



(B7) 



(B8) 



We combine these equations to determine a second order equation for x 
as follows 



d 2 x 

di 2 



= « * p- = 

R V dx 



/W ,,2 w , 
-(- <J>) X + - <pw 



d 2 x 

dT 2 



,w .2 w 



or 
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The solution to the above equation is given by 

x(t) = A cos — 4>x + B sin — <f>T + — , 

K K 9 

where the constants, A and B, are determined by 

x (t = 0) = £ sin s = A + t l -> A = it sin s - ^ 

<P <p 

~ ^ (x = 0) = ^ (p y (t - 0) = ~ <p £ cos s=^-cj)B->B=£ cos s . 
a! k k k 

Hence, after some simplification, we obtain: 

x(t) = H sin (s + <f>t) + -7 (1 - cos ^ <f>i) . (B9) 

K (p K 

Similarly, it may be shown that 

y(x) = i cos(s + — (f)i) + y sin (pr . (BIO) 

K 9 K 

Note that the above equations (and also (B7) and (B8) hold from the terminal 
surface (i = 0) until a transition surface is reached. 

Transition Surfaces and Termination of the Barrier . 

We have seen (equation ( B3 ) ) how the "steering” variable (p (this 
variable determines the rate of change of heading for the moving point in 
the fixed coordinate system) is determined by 

(p = sgn A (t ) , (B3) 

where A(t) = p^y - p^x . There is a problem, however, at t = T , since 
A(t = T) = A(t = 0) = sin s(£ cos s) - cos s (i sin s) = 0 . We can overcome 
this by a continuity (with respect to t) argument. Moreover, we have 
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dA 

dx 



% 

dx 



y + p 



_ 

1 di 




“ P 



dx 
2 dx 



= <p 2 |*)y + Pl c- f +x 



+ w) - C- P;L ^4>) - P 2 (^ y<f>) 



Hence, 

= p w = w sin(0 + cj> t ) with A(t = 0) = 0 , 

d T i R 

and then we finally obtain 

A(t) = T {cos s - costs + —■ <J> t ) } . (Bll) 

<p R 



Thus, for 0 _< x 


<_ x^ (where 


T i 


will be determined presently) , 




{ -1 for - 


7T 

— < 


s < 0 


<j)(x) = 




2 — 






+1 for 


0 < 


TT 

S ±2 . 



Since optimal trajectories are symmetric about the y-axis, we only consider 
solution behavior in the right half-plane of our relative coordinate system. 

7T 

Consequently, for 0 < s <_ y , we determine as follows: it is the 

first time that A(t) = 0 after t = 0 . This happens when 

cos s = cos (s + ~x^) 9 

which is precisely when 

o . w 

2tt - s = s + — t . 

R i 

Hence, for 0<s<_Tr/2,we have 

t 1 = — (jt - s) . (B12) 

1 w 

Equation (B12) determines such that cj) (t ) = 1 for 0 < t £ . For 

i > , we have A(t) < 0 (we have not proved this) and hence 4> (t ) « -1 
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on optimal trajectories in some interval past . Hence, the locus of 

such points where (B12) holds defines a transition surface. 

For s = tt/2 , we have the barrier and hence <(>(t) =1 on this "right 
barrier" for 0 jb T Rtt/w • From (B9) and (BIO) , the equations of the 
barrier are 



x(t) - R = -(R - £) cos ~ t , 

R 

y(x) = (R - £) sin ^ x , 



which we may write as 



/ \ -n / w w 

x(t) - R 1 cos — t sin — t Z 

R R 

= | 

1 / \ / . w w : 

y(x) J \ -sin - x cos - x / 

\ 

that 

This equation says f to determine a point of the barrier at 

i« 

we rotate the point x=£-R,y=0 through an angle 

negative (counter-clockwise) sense about the point x = R 

for 0 < t < — 7T , the barrier traces out the curve shown 
— — w 



- R 
0 

R 

time t < it , 

— w 

— x in the 
R 

, y = 0 . Hence, 
below in Figure B1 . 




Figure B1 . 



Barriers to Destroyer to Fixed Destination Problem. 
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Now we shall explain why this barrier must terminate at x^ = tt2/w. 

We have previously discussed this subject in section Illb of the main text 
(see also pp. 210-214 of [30]). To summarize, there are two equivalent 
criteria for termination of the barrier: 

(1) (Isaacs) due to a change in the orientation (direction of travel) 
of the barrier curve this semipermeable surface cannot be 
extended , 

(2) (Taylor) the barrier terminates if its extension would lead to a 
"multicovering" of extremals in the region of extension (with 
the extension being non-optimal) . 

For the problem at hand, for t < x^ = ttR/w and s = tt/ 2 there would be 
a cusp (tangent to curve continuous but both dx/dx and dy/dx change 
sign) , since the extension (we have not proved this) would be an arc of a 
circle with center (-R,0) with <J> = -1 , I.e., curve "changes orientation 
and there can be no semipermeable continuation" ([30] p. 211). We recall 
that for x = x(x) and y = y(x) 

d£ 

dy _ dx , 
dx dx_ 

dx . 

Thus we can have the tangent, dy/dx , continuous but both dx/dy and 
dy/dx change sign. The curve has a cusp at such a point. For the problem 
at hand, the continuation of the barrier past x^ would produce such a 
cusp, but a cusp is not necessary for termination (see homicidal chauffeur 
game Appendix C) . 
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We might also argue along the lines of (2) above that if (in the 
game of degree) the barrier were to be extended, it would intersect ad- 
jacent extremals causing a "mul t i-valued M solution. Hence, we discard the 
dashed portion shown in Figure B1 . 

Continuation of Solution Past Transition Surface 

Continuation past = 2R/w(tt - s) for 0 <_ s < tt / 2 , results in 

an arc of a circle with center (-R,0) with travel for increasing t in 
positive (counter-clockwise) direction, i.e., 4> = —1 Time has not 

allowed all such analysis details to be worked out. 

The Singular Solution (Universal Surface) 

Fcr the problem at hand, the Hamiltonian is a linear function of 
the control variable <p , 



of <p vanishes for a finite interval of time. In this case we have a 
singular solution [32], [33] for which the necessary condition of maximizing 
the Hamiltonian (with respect to the control variable <\> ) does not provide 
us with a well-defined expression for the extremal control. Isaacs 
([30] Chapter 7) uses the terminology universal surface . 

A singular extremal is determined from the conditions [32], [33] 



H (t , x , p , <(>) = <t> ^ (p 2 x - p^y) + (1 - p 2 w) 



(B14) 



with the control being determined by (B3) except for when the coefficient 




and 



_d_ 3H 

d t 9<{) 



0 , — = 0 , etc. as needed 
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For the problem at hand we also have H(t,x,p,(j)) = 0 , since the termination 
time is unspecified. Also, above equation (Bll) we saw that dA/dx = p^w . 
Thus, since 3H/3<|> = - ~ A(t) , we have the following equations for a 
singular subarc 

1 - p^w = 0 , 

|(p 2 x - Pl y) = 0 , 

2 

and p^ w /R = 0 . 

Hence, we see that on a singular subarc we have 



P 2 (t) = l/w > 0 , 
P x (t) = 0 , 

x(t) = 0 . 



(B15) 



The singular control is determined by the above conditions that the dual 
variables are constant for a finite interval of time. Recalling that 
dp^/dt = -(fjp^w/R and dp^/dt = <J>p^w/R , we see that <j)(t) = 0 is the 
required singular control. A physical interpretation is enlightening: 
once the destination is straight ahead (xtt) = 0) , the destroyer steers 
a straight course (^(t) = 0) . 

We must further test to see if this singular solution can yield the 
optimal return; i.e., minimum time. A necessary condition for a singular 
subarc to yield the minimum return [34] is (see also [30] pp. 187-188) 
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2 

We have already shown that d/dt (3H/3<j>) = p^w /R . Hence 




— 4>p 2 w ^ w / r ) 



2 



Thus 



_3_ 

3(f) 




-p (w/R) 



2 



(B16) 



But on the singular subarc (B15) must hold, so = l/ w * Hence, on 

the singular surface 



and the necessary condition is met. 

c . The New Geometrical Construction for Optimal Trajectories . 

The following is given to show our original motivation in developing 
this geometric interpretation of the solution to the state equations. 

Others may prefer the analytic geometry of transformations of ordered 
pairs, but we usually remember such things by considering complex numbers. 

We start by recalling ([11] p.8) the well-known geometric interpretation 
of complex numbers in which a complex number is represented by a point in 
the plane. With this interpretation, we may then develop that multiplication 
by a complex number of unit modulus corresponding to a rotation. We now look 
for an algebraic representation. It is well-known that the field of com- 
plex numbers is isomorphic to a field of 2x2 matrices with the correspon- 
dence being given by 




0 , 



a 



-b 



a + ib 



a 
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Thus, we see that the equation (giving the locus of the transition surface, 
see above) 

f x ( T i ) - R /cos 2s -sin 2s\ £ sin s - R \ 

= [ ' ! ! (B17) 

y(t^) sin 2s cos 2s ; \^£ cos s / , 

is equivalent to (see pp. 2-5 in [11]) 

i2 s 

(x(t^) - R) + iy(i^) = e [ (£sin s - R) + i£ cos s] , 

and thus represents a rotation of the point x = £sin s , y = £cos s 
through an angle 2s in the positive sense (counter-clockwise) about the 
point x = R , y = 0 . After we recognize this, we may, of course, use 
analytic geometry to reach the same conclusion. 

Thus, we consider optimal trajectories in the right half-plane for 
0 £ T £ = 2 R(tt - s)/w. We may write equation (B9) as 

x(t) - R = (£ sin s - R) cos ^ t + £ cos s sin ^ t . (B18) 

K R 

Similarly, equation (BIO) becomes 

y(i) = £ cos s cos ~ t - (£ sin s - R) sin ^ t. (B19) 

We may write the above as 

for 0 s 7T / 2 and 

/ \ -n / w 

. xCt) - R i cos ” T 

i 

\ / V . W 

\ y (t) \-sin — T 

where 

x(T) = £ sin s , y(T) = £ cos s. 



0 <_ 1 _< = 2R/w(tt - s) 



w 

Sin R T \ 



x(T) - R 



I 



w 

COS — T 
R 



i \ y (T) 



(B20) 
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The geometric interpretation of (B20) is that the point x(t) > y (t) 
which lies on a trajectory terminating at x(T), y(T) may be obtained by 
rotating the point x(T) , y(T) through an angle wx/R in the negative 
sense about the point x = R , y = 0 . We recognize the example above 
(B17) as the special case of (B20) when t = = 2R(tt - s)/w. 

Let us see how the above geometric interpretation is useful in 
sketching the transition surface. From (B12) we see that a change in 
steering occurs at different times for different trajectories. We have 



0 


w 0 

“ T _ = 2 TT 




R 1 


tt/4 


3 / 2 TT 


TT / 2 


TT 



By considering the geometric interpretation of (B20) , we obtain the 
picture shown below in which the transition surface is a dashed line. 




56 



d . Geometry of the Solution 

In this section we summarize the results of analysis by drawing a 
picture of the optimal trajectories with the control, <j> , being indicated 
in various regions. The singular subarc, x = 0 , and paths leading to 
it (we have not done this analysis) are also given. This is shown in 
Figure B2. Again, the reason this "one-sided" problem has been 
studied in such detail is that it is the limiting case of the homicidal 
chauffeur game when the Evader is immobile, i.e., = 0 . 
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Figure B2 . Optimal Trajectories for Destroyer to Fixed Destination 



57 



We should note that the above solution is in the relative coordinate 
system. We discuss briefly how the optimal paths look in a fixed coordi- 
nate system. Such is shown below, where the point P is the movable point 
and three destinations, 



A, B, and C are shown. These points correspond to A, B, and C of 
Figure B2. Point A is reached by turning as sharply as possible, <J> = 1 . 
Point B is reached by turning sharply and then a straight-on approach. 

The last part is the singular subarc where <J> = 0 . Point C is within 
the minimum turning radius, R, and some maneuvering is required by P . 



B 




t A 



P 
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APPENDIX C 

ORIGINAL HOMICIDAL CHAUFFEUR GAME 

In this appendix we derive parts of the solution to Isaacs 1 homicidal 
chauffeur game [30] . We do so because of its close similarity to the sur- 
veillance-evasion game (see section III of main text) . 

We state the problem and present details of its solution. Next, we 
discuss our new geometrical construction for optimal trajectories in this 
problem. Finally, we discuss the geometry of the solution and summarize 
the analysis results. A more detailed discussion of many solution steps 
is to be found in Appendix B. 

a. Statement of the Problem . 

The problem is to determine how a Pursuer, who travels at a constant 
speed w , with a minimum turning radius R , should steer to capture 
in minimum time T an Evader, who travels at constant speed w^ , has no 
restriction on maneuverability, and tries to maximize the capture time T. 
Capture conditions are defined by Pursuer and Evader being separated by a 
distance i . In a stationary coordinate system the problem is 

f T 

min max dt with T unspecified , 

<j> 6 0 

subject to: dx-^/dt = w^ cos a , 

dy^/dt = w^ sin a , 
da/dt = cj> w^/R where -1 <_ cj> <_ 1 , 
dx^/dt = w^ cos 3 



dy^/dt = w 2 sin 3 , 
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with initial conditions 

x 1 (t=0) = x^ , y 1 (t=0) = , a(t=0) = a Q 

x 2 Ct=0) = x° , y 2 (t=0) = y° , 

and terminal conditions 

[x 1 (T) - x 2 (T)] 2 + [y x (T) - y 2 (T) ] 2 = £ 2 . 

We transform the above problem to the relative motion coordinate system of 
Appendix A in the same fashion as shown there to obtain: 

f T 

min max dt with T unspecified , 

* * 0 

subject to: dx/dt = -w^/R y<j> + w^ sin ip , 

dy/dt = w^/r x(J> - w^ + w^ cos ip where ”1 jL <j> ^ 1 > (Cl) 



with initial conditions 

x (t=0) = x Q , yCt=0) = y Q , 

and terminal condition 

x 2 (T) + y 2 (T) = i 2 . 

b . Development of Solution . 

Hamiltonian, H(t,x,p ; <p 

W 1 

H(t,x,p;<j> ,t|0 = 1 + Pj^C- Y~ y 4* + w 2 sin ^ 

W 1 

+ p 2 ^ ~R~ X<t> ~ W 1 + W 2 C ° S ^ ’ 



(C2) 
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where p^(t) = 3 J*/ 3 x (t) and 



rT 



J* = min max 

(j> \jj 



J 0 



dt = max min 

\l> <j> 



p 2 (t ) = 3 J*/ 3 y (t) are dual variables and 
T 

dt . We determine extremal strategies 

0 



from 



and 



Hence 



where 



min max H(t >x,p ; cj) ,t|;) •* 

W 1 

min {- — - P 2 x H > 

<}> 

max {v^Cp^ sin ip + p 2 cos '/')) • 



4 > = sgn A(t) , 
A(t) = p x y - p 2 x . 



(C 3 ) 



To maximize with respect to ^ it suffices to consider 
f 0|0 = p^ sin ip 4 * P2 cos 1 p . 

A necessary condition is that 

df/di^ = 0 = p^ cos \p - P2 sin \p } 



and hence 

tan 41 = p 1 /p 2 

Thus, to maximize we must have 



sin ^ 




cos 




+ P 



2 

2 



(C 4 ) 
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Boundary Conditions for Dual Variables 

—y f \ ~y 

We must have p = (p^ p_) parallel to the normal, n , pointing into 



the state space to the terminal surface. Now, since 
n = sin s i + cos s j , we have 

p. (t = T) = sin s , p (t = T) = cos s 
1 w 

Usable Part of Terminal Surface 

We have that 

n = sin s i 4- cos s j , 



where the terminal surface is parametrically represented by s as shown 
below. ^ 

y 



w_ w_ 

• 1 -> j- -y 

X = (- Y" + w 2 sin + ^"r - x< ^ - + w 2 cos > 




\ 



V 




Hence, we have 



x(t = T) = i sin s and y(t = T) = i cos s. 



n • X(t = T) = ( <fr i cos s + w sin ip)sin s 

K Z 




Hence , 
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and when we recall that 

P L (t = T) 

sin ip(t = T) = - — — = sin s , 




P 2 (t = T) 

cos ip(t = T) = — = cos s , 




we obtain 

- v 

n • X(t = T) = w 2 ” cos s S ® » (C6) 

where we also assume w^ < (otherwise capture is impossible unless the 
Evader is stupid). Hence, the useable part of the terminal surface is 
given by 

UP = {s|~S <_ s <_ S where cos S = w^/ w^ , 0 <_ S < tt / 2 } . (C 7 ) 



The Adjoint Equations and Their Solution 
We have that 

W 1 

d P]L /dt = - 3H/3x = “P 2 * 

W 1 

dp 2 /dt = - 9H/3y = Pl — <J> 

To integrate backwards from time of capture, T , we let t = T - t 
to obtain 

W 1 

dp^/dx = P 2 $ P-^* = °) = sin s » 

W 1 

dp 2 /dx = - p — <p p 2 (t = 0) = cos s 
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We combine these equations to obtain a second order differential equation 
for as follows 



,2 /, 2 dp 2 
d p i /dT = TT 



W, 



R 






= -p x ( 



R 



4>)‘ 



or 

2 2 w i 2 

d p^dt +( — <{)) p x = 0 

The solution to the above equation is given by 

W 1 W 1 

P 1 (t) = A cos — (j)T + B sin — <j>x , 



where the constants, A and B , are determined by 



p^(x = 0) = A = sin s , 

w w w 

dp^d-rCt = 0) = — 4>B = p 2 (x = 0) = — 4> cos s , 



or B = cos s 

Hence, after some simplification, we obtain: 



w„ 



P 1 (x) = sin (s + — <J)T ) 



Similarly, it may be shown that 

W 1 

P 2 (x) = cos(s + — <j>t) . 

Taking note of equation (C4) , we also obtain 

W 1 W 1 

sin = sin(s + — (j>x) and cos \p = cos(s + — $x) . 

R K 



(C8) 



(C9) 



(CIO) 



64 



Solution to the State Equations 

In the "backwards time" we have 



W 1 W 1 

dx/dT = “jf ~ w 2 sin ^ s + 4 >t) 



:(t = 0) = £sin s , 



W 1 W 1 
dy/dx = - ^pj)X + cos(s 4- — <j>x) 



y (x = 0) = cos s 



We combine these equations to determine a second order differential equation 
for x as follows 



d x 



2 w w w 

/dt = — <t> - w 2 — (J) cos(s + — <f>x) 



R 



w 2 W 1 W 1 

= - (-g- <f>) X - -Y ^{w 2 cos(s + Y" _ W l^ 



W 1 W 1 
w 2 y + cos ( s + y ^ 



or 



2 2 W 1 2 w l W 1 w l 

d x/dx + (— <t>) x = -2 y <f> w 2 cos (s + — <{>T ) + — 



(Cll) 



with 

x(x = 0) = i sin s , 

W 1 

dx/dx(i = 0) = — <j> £ cos s - w sin s 
K Z 

The general solution to the above inhomogeneous differential equation is 
given by 

x(r) = X (x) + X (x) , (C12) 

h p 



where 
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x^Ct) is the general solution of the homogeneous equation, 

Xp ( t ) is any particular solution to the inhomogeneous equation. 

The general solution of the homogeneous equation is given by 

w w 

x (x) = A cos — (j)T + B sin — <j>x . (C13) 



We use the method of variation of parameters ([20] pp. 72-73) to 
find a solution to the inhomogeneous equation. (This is quite messy but 
I have not found an easier way.) We use two linearly independent solutions 
to the homogeneous equation. 



U-^x) 


= cos 


i 

R 


<j>x , 


u 2 (x) 


= sin 


R 


4>x , 



and determine v^(x) and v 2^ x ^ where the particular solution is assumed 
to be of the form 



x p (x) = V 1 (t)u 1 (t) + V£ (t)u 2 ( t ) • 



(C14) 



Hence 



dx dv dup dv^ du^ 

dx dx 1 1 dx dx 2 2 dx 



Now we set 



dVf dv 

~d7 u 1 + ~d7 u 2 



0 , 



(C15 ) 



leaving 



dx 



d u 



du„ 



1 , ““2 
dx V 1 dx V 2 dx 



66 



so that 



2 2 
d x /dx 
P 



dv^ du^ d^u 

dx dx V 1 dx^ 



1 ^ dv 2 du 2 
+ "dT' — + V2 



,2 

d u. 



dx 





. dv l d “l dv 2 

dx dx dx 


du 2 

dx 


w 

1 0 

C TT* } (v 1 u 1 


+ v 2 o 2 ) , 


since u^(x) 


and u^C.t) are solutions 


to the homogeneous equation. 


Hence, using 


(C14), 








d 2 X 

7i + < 


”l .2 . dV l du l 

R ^ X p dx dx 


+ d ? 


du 2 

dT - R(T) 


5 


where R(x) 


is the right hand side of 


(Cll) and 






w 

R(t) = -2-j~ 4> w 2 cos (s + 


2 

w w 

X +t) +t ♦. 


To summarize. 


(C15) and (Cl6) give us two equations 


for dv^/dx and 




du i dv l du 2 

dx dx dx 


dV 2 

dx 


R(t) 






dv dv. 










U 1 i + U o i 

1 dx 2 dx 


= 0 . 






Solving these 


equations by Cramer T s rule, we find that 








dU ? 






i 


R(t) 


Z 

dx 






dv i 


0 


u 2 


R(t)u 2 (x) 




dx 


du i 


du 2 


“ W1 

<'T*> 






dx 


dx 








u i 


u 2 





(C16) 

(C17) 
dv^/ dt , 



(C18) 
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Similarly, 




R(t)u 1 (t) 

<r*> 



(C19) 



Substituting (C17) and the definitions of and into the above, 

we find that 



dv„ 



dx 2 



W 1 W 1 
w 0 sin(s + 2 — <J> t ) - sin s ~ w^ sin 4>x , 



and 



dv^ 

dx 



w i w i 

-w cos s - w„ cos(s + 2 — 4 >t) + w cos 4 >t . 

■<- K i R 



Integration of these equations yields 



W 2 R W 1 R W 1 

v 1 (x) = - 2w ^ cos(s + 2 — 4>x) - tw 2 sin s + — cos — 4 >t , 



(C20) 



and 



W 2 R W 1 R W 1 

v 2 (t) = - tw 2 cos s - — sin(s + 2 — <f>x) + - sin — 4> x . 



(C21) 



Substituting (C20) and (C21) and the definitions of u^ and u^ into 
(C14) , we obtain after some manipulation 



w 2 r ^ R 

x p (x) = - 2^7 cos(s + T 4,T) ' TW 2 sin(s + T * t) + J 



(C22) 



Combining (C13) and (C22), we see that the general solution to (Cll) is 



given by 
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W 1 W 1 

x(x) = A cos — <J>x + B sin — <p T 
K K 



w„R w 

2^7 Rc °s (s+ T^ t) 



w. 



. / , 1 i \ , R 

TW^ sin(s + <pT ) + — 



(C23) 



The initial conditions to (Cll) lead to the following determination of 
the constants in (C23) : 

A , . x V* r 

A = £sin s + - — - cos s — 7 - , 

2 w n <p (j) 



B = £cos s - 



w^R 
2 w. d> 



sm s . 



(C24) 



Substituting (C24) into (C23) , we obtain after some manipulation, 

W 1 R W 1 

x(x) = (£ - iw 2 )sin(s + — <J>t ) + — (1 - cos — <|>t) . 



(C25) 



Similarly, we may also obtain 

R 

y(i) = (£ - xw 2 )cos(s + — 4>x) + — sin — (j)T. (C26) 

Note that the above equations (and also (C 8 ) , (C9), and (CIO)) hold from 
the terminal surface (t = 0 ) until a transition surface is reached or 
the trajectory terminates. 

Equation of the Barrier 

We have seen (see Appendix B) that the barrier (boundary of the domain 
of controllability) is an optimal trajectory which terminates at the BUP. 

We have seen in (C7) above that the boundary of the use±>le part (BUP) is 
given by s = +S , where 0 <_ S < tt/ 2 and 



cos S = w^/w^ . 



(C27) 
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Because of the symmetry about the y-axis in this problem it suffices to 
consider the right barrier, i.e., the one for which s = S . In the next 
section we show that <f>(i) = 1 for s = S and 0 < t < where is 

also determined. Thus, the equation of the right barrier is given by 

for 0 <_ t <_ 

w w 

x(t) = (Z - TW 2 )sin(S + + R (-l “ cos — x ) , (C28) 

and 

w w 

y (t ) = (i - tw 2 )cos(S + — t) + R sin — - t . (C29) 

It is not obvious (as Isaacs and others seem to infer) that (C28) and 
(C29) are the equations for the involute to a circle. It, therefore, 
seems appropriate to digress and review some analytic geometry. Our dis- 
cussion follows R. Courant ([12] pp . 280-283 and pp. 307-310). 

Consider a curve represented parametrically by x = x(t) , y = y(t) 
in the plane. The curvature, < , of this curve at a point is given by 
k = da/ds , where tan a = y/x = (dy/dt)y // (dx/dt) . The radius of curvature 
is defined by p = 1 /k . For a given point on the curve, there is a 
circle of curvature corresponding to the point. This circle touches the 
curve at the point and there has the same sense of description and the same 
curvature as the curve. Its center is called the center of curvature. Con- 
sider the diagram below. At any point (x,y) , the center of curvature, 
(£,n) > is given by 
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The locus of such centers of curvature to a curve is called the evolute 
of the curve, C . We further call C the involute of its evolute. The 
evolute is the "envelope" of the normals to C . An important fact that 

we shall use later is that the tangent to the evolute of a curve is normal 

# # • 

to the curve, i.e., £ x + n y = 0 

If we have a curve £ = £ (o') , n = n (o') where a is parameter, then 
the equations of the involute to this curve are given by 

x = £ 4- (a - a)£ , 
y = n + (a - o)n 

For a circle, represented parametrically by 

£ = -cos t , 

(C30) 

n = sin t , 

the involute is given by 

x = -cos t - t sin t, 

(C31) 

y = sin t - t cos t . 

It is worth noting that all the normals to the curve given by (C31) are 
tangent to the circle (C30) . We show the curves below. Another geometric 
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property of involutes that we shall use is that: let, 



C 




= distance from A to B , 

&2 ” arc length from B to D , 

= distance from C to D, 

where 

AB and CD are normals to involute. 

Then 

*1 + = S (C32) 



To show that (C28) and (C29) are the involute to a circle, we must 
show that they are of the form (C31) . To do this we consider 



w w 

R cos t = R cos (— t 4- S - S) 
R R 



w w 

= R cos (— t + S)cos S + R sin(— x + S) sin S 
R R 



Considering (C27), we see that 



w. 



w„ w. 



w. 



R cos t = R — cos t + S) 4- — X; “ - w 2 sin (-~ t + S) . 
R w R w^ 1 2 R 



Thus, (C28) becomes 



w ? W l R 7 2 2 

x(t) - R = - (R — )cos(— t + S) +{[£,- — vw - w 
W^ R W^ -L Z 



w w 

) - < R ^ > T 



w„ 



sin x + S) 



(C33) 



Similarly , 
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w ? w i r ; — 2 — 

y Or) = (R — ) sin (~r T + s) + {[* - — / w/ - 

W.. K W_ ± 



W r 



W 2 W 1 

J - (R ^ ) T T} 



w i 

• cos (— x + S) . (C34) 

K 

Considering (C30) and (C31) , it is clear that (C33) and (C 34 ) are the 
equations of an involute to a circle of radius RCw^/w^) and with center 
x = R , y = 0 . The equations of this circle are 

W 2 W 1 

£ - R = - (R 7T ) cos (~ t + S) , 

W- K 



W W 

n. - (R — ) sin (— 



T + S) , 



where the involute is unwound from 





u 




] < 0 
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To insure that the involute is unwound under all circumstances, we 
show that the quantity in brackets ahove is always negative. This 
from elementary geometry considerations in the above figure. 



or 



£ 2 + (R^) 2 < R 2 
W 1 



£ 




for all values , 



Transition Surfaces 
We recall that 

<t>(x) = sgn A(t) 

where A(x) = p^y - P 2 X • 

Recalling that 

x(t = 0) = £ sin s , y(x = 0) = £ cos s, 
and P 1 (x “ 0) = sin s , p 2 (x = 0) = cos s , 

we see that 

A(x = 0) = 0 



Also , 



dA/dx = 



d Pi 



y + p r 



iz . th. 



dx 

dx X ' P 2 d7 



dx J ' *1 dx 

into which we substitute the state and adjoint equations to obtain 

W 1 

dA/dx = p.,w, = w- sin (s + — x ) with A(x = 0) = 0 . 

Ill K 

Integration of the above yields 

W 1 

A (x ) = ^ {cos s - cos(s + — <j>x) } . 



must 

follows 



(C 3) 



(C35) 



(C36) 
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Thus, for 0 <_ t (where will be determined presently). 







-1 


for 


-S £ s < 0 


<f> (t) 


= 


1 — 1 
+ 


for 


0 < s <; S 



(C37) 



where S is determined by (C27). By the symmetry of the problem, we 
concentrate on the right half— plane* Consequently, for 0 < s <_ S , we 
determine as follows: it is the first time that A(t) = 0 after 

t = 0 . This happens when 



w„ 



cos s = cos(s + — T-) 
R 1 



which is precisely when 

2 7T - s 




1 * 



Hence, for 0 < s <_ S < tt/2 , we have 

t = — (tt - s) . (C38) 

1 w x 

Equation (C 38 ) determines such that (*0 = 1 for 0 £ t <_ . 

For t > , we show later that A(t) < 0 and hence = -1. If 

trajectories do not terminate before condition (C38) holds then the latter 
gives a transition surface. 

For s = S , where cos S = w^/w^ , we have the barrier and hence 

=1 on this "right barrier" for 0 <_ t <_ 2R/w^ (tt - S) ♦ From (C28) 

and (C 29 ) , we may write the equations of the barrier 

W 1 W 1 
x(t) - R = (£ - twJcos S sin — t + { (i - iw 0 )sin S - R} cos — t, 

Z R Z R 

w i w i 

y(x) =-{($,- xw 0 )sin S - R} sin — t + (£ - tw„)cos S cos — t, 

Z R Z R 



75 



which we may write as 



/ x(t) - R \ 





(£ - Tw^sin S - R 




w 



(C39) 



\ yCO 



/ 



COS 



R 



1 



T 



\(l - xw 2 ) cos S / 



The geometric interpretation of (C39) is as follows. To determine a point 
on the barrier at time t <_ (2R/w^) (tt - S) , we start at the point x = £sin S, 
y = kos S on the terminal surface (this is the BUP) and move to a point 
which is a distance xw^ along the straight line connecting the first point 
to the origin ( and toward the origin). We now rotate this latter point 
through an angle (w^/R)x in the negative (clockwise) sense about the 
point x = R , y = 0 . Hence for 0 <_ (w^/R)t <_ 2tt - 2S , the barrier 
traces out the curve shown below in Figure Cl. 




x 






Figure Cl. Barrier in Homicidal Chauffeur Game. 
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Continuation of Solution Past Transition Surface and Termination of Barrier 

Later we show that paths terminating on terminal surface for 0 < s < S 
may all converge to the same point and hence terminate there (except for the 
barrier). The time of this conveyence depends on the speed of the Evader, 
w^ . We recall that in the limit as w^O the solution approaches that 
shown in Figure B2 . If w ^ is small enough, the transition surface beyond 
the termination of the barrier may be reached by a trajectory before the 
trajectory terminates by reaching the ,r focal point." Hence, we consider 
extension of the barrier which we know continues until x^ . Details for 
other trajectories (if they exist long enough) are similar. 

Thus, we consider the continuation of the right barrier past 
(w^/R)t^ — 2 (tt — S ) . From (C8) and (C9), we have 

P 1 (x 1 ) = -sin S , p 2^ T 1^ = COS S * (C40) 

From (C25) and (C26) , we obtain 



x(x ) = (& - x w ) (-sin S) 4- R(1 - cos 2S) , 

1 1 Z (C41) 

yCx 1 ) = (£ - T l w 2^ COS S - R sin 2S . 

Later, we show that 4> (t ) = -1 for t > x^ . We assume this for now. The 
adjoint equations for t > x^ are 



dx 



w i 
P 2 R 



dp. 



P 1 R 



P 1 Cx 1 ) = -sin S , 



dx 



p 2 C T i ) = cos S 
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The solution to the above equations is (for x > x^ ) 



W 1 

p^CO = -sin(S 4- — Ct - t^)) = sin ip , 



w 



P 9 (t) = cos (S + — (t - t )) = cos <Ji 



The state equations for r > x^ are 



W 1 W 1 
dx/dx — y + sin(S + — (x 



t 1 )) . 



W 1 W 1 
dy/dx = “r” x + w i ” w 2 cos ( S + ( T " T ]_^ » 



(C42) 



(C43) 



with initial conditions given by (C41). A rather laborious computation 
(we omit the details) yields the solution to the above as (for x > t^) 

W 1 

x(x) = - Ci - xw 2 )sin(S + — (x - x^) 

w w 

+ R{ 2 sin — (x - x 1 ) - 1 - cos (2S 4 - — (x - x^))} 



and 



W 1 

y (x) = (l - xw 2 )cos(S + — (x - T^) 



w w 

+ R{ 2 sin — (x - x^) - sin(2S + — (x - x^))} 



(C44) 



We may write equations (C44) as (for x > x^) 

w, w. 



x(x) + R \ /cos — (x - x^) -sin — (x - x^)\ ' (& ~ ™^) (-sinS)+2R-Rcos2S 



1 I 

i I I w w 

\ y(t) I \sin — (x - x L ) COS — (x - x^)/ \(Jl - xw 2 )cosS-Rsin2S / 



(C45) 
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or using (C41) 



w w v 

I x (t) + R\ cos -sin — \ xCxp-H^Cx-xpcosCy -S) + R 



\ 



^ W 1 w i j 

\ y (t) j \sin — tx-x^ cos — y w Cx-x^ sin (~ -S) / 

(C46) 



The most important aspects of the geometric interpretation of (C46) are 
that rotation is in the positive sense about the point x = -R , y = 0 
and the involute continues to unwind. That (C44) does indeed trace out 
an involute may be seen by writing it in the equivalent form 



x(x) + R = R — cos(S + -^ (x - x. )) 
w^ K i 

+ [{3r/i - (w„/w n ) 2 - £} + (R — ) (t )]sin(S + — (x - r )) , 

Z ± W^ K K 1 

/ 2 w 2 W 1 W 1 

y(x) = [-{ 3Rvl - (w ? /w ) - £} - (R — ) (t ) ] cos (S + — (x - x )) 

^ 1 w- R R 1 



+ 




w„ 



sin (S + — (x - x 1 >) 



(C47) 



which is an involute to a circle of radius R(w 2 /w^) and with center 
x = -R , y = 0 . The involute is unwound from 



w w r- 

( x) (R — ) = -3R/1 - (w./w.)‘ + £ < 0 . 

K W^ Z J. 



If we were to try to extend the barrier given by (C44) (or any equivalent 
form), we would find that the barrier "bends back on itself" as shown by 
the dashed line in Figure Cl. Hence, by the arguments given in Appendix B, 
the barrier must terminate at = (£R/w^) (tt - S) 
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We also note that if the barrier were extended for x > x , then 
the tangent to this curve would be discontinuous. We denote 



dx 

dt 



T = T + as bein 8 the limit as x+x^ and t _> . Similarly for 



d£ 

dt 



T=T _ 



We may show that for the barrier we have 



dx 

dt 



T=T . 



= w^{ sin S - (i - x -^w^) /R} cos S , 



dt 



T==T ~ = w^fsin S - (£ - i^^/Rlsin S , 



and hence (as we knew before) 



d£ 

dx 



1=1, 



tan S 



Also 



dx 

dt 



+ = -w^{3 sin S ~ (£ - x^w^/Rjcos S , 



and 



d^ 

dt 



T=T , 



+ 



dx 

dt 



X = X . 



Hence if x w > £ (causing "focal point" in field of trajectories, i.e., 

1 Z the 

all paths excepti barrier terminate before x = x ), both dx/dt and 
” The 1 

dy/dt change sign at x = x^ . |curve would "almost have cusp" at x^ if the 
barrier were extended (except for dy/dx being discontinuous). 
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The Singular Solution (Universal Surface) 

Since the Hamiltonian is a linear function of the Pursuer's control 
variable <j> , the maximum principle does not determine the control when 
the coefficient of <j> vanishes for a finite interval of time (see Appendix 
B) . Part of a trajectory for which this occurs is called a singular subarc. 
The Hamiltonian is 
W 1 

H(t,x,p;<j>,4>) = <j>— (p 2 x - p^) + w 2 (p 1 sin \p + p 2 cos t|>) + 1 - p^ 

or using (C4), we have 

W 1 7 2 2 

H(t ,x, p ; <j> ,i|j) = <{>— (p 2 x - p 1 y) + w 2 /p 1 + p 2 + 1 - p^ . (C48) 

We determine the conditions for a singular subarc from 

H = 9H/9<j> = (d/dt) (9H/9<j>) = 0 (C49) 



Recalling that above equation (C36) we had dA/dt = -w^p^ and noting that 
9H/9<t> = (-w^/R) (A(t) ) , we have from (C49) the following conditions for 
the singular surface 

7 2 2 

W 2 /p l + p 2 + 1 " P 2 w i = 0 ’ 

W 1 

T (P 2 X " P l y) = ° ’ 

2 



and hence 



P x (t) = 0 , 

p ? (t) = „ h > 0 ’ (C50) 

2 w ! ~ w 2 



x(t) = 0 
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The singular control required to yield the above is readily seen to be 

J 2 " ' 2 

<KO = 0 . Hence recalling (C4), i.e., sin ip = p 1 (t)//p + p 2 , 



we see that i|/(t) - 0 on the singular subarc. We have not traced paths 
which lead to x(t) = 0 backwards as Isaacs has done ([30] pp. 193 - 194). 
The necessary condition for optimality [34] on the singular subarc is also 
met since 




c ^ » 

^ d<j> ’ 



-(w^/R) 2 < 0 . 



Determination of Capture Criterion 

We have discussed in section Illb of the main text that a very impor- 
tant question is whether or not the barrier divides the state space into 
two parts. For the problem at hand, if the barrier does not divide the 
state space into two parts, then (it appears as though) capture can occur 
from any initial point, i.e., the entire state space is the capture zone. 
The only way that the barrier can divide the state space into two parts is 
for the "left" and "right" barriers to meet in the y-axis. We now develop 
the condition for this to occur. 

We consider Figure C2 and recall the relationship between two normals 
to the involute of a circle given by (C32). When capture can be avoided, 
the barrier intersects the y-axis. In Figure C2, we have 

£ 1 = l 2 + *3 ’ (C51) 
i.e. the difference in the length of the normals is equal to the distance 

on the perimeter of the circle (evolute) between points of tangency. 

For capture to always occur, i.e., barrier does not intersect or touch 
y-axis, we must have 

R > i . 



(C 52) 
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y 



4 








/ 



Figure C2. Determination of Capture Criterion. 
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We also have 

sin S = £ 4 /R = (2 + H 2 )/R , 

or 

l 2 = R sin S - 2 , (C53) 

and 

2^ = R(w 2 /w 1 )( y - S) . (C54) 

TT 

Now, since (y - S) = ^ 3 W 1 / (R w 2 ) > we have (also using (C27 ) ) 

sin(£ 3 w 1 /(Rw 2 ))= cos S = vy/w , 

and hence 

^3 = r ^ w 2 //w i^ ) sin 1 (w 2 /w 1 ) (C55 ) 

Combining (C51), (C52) , (C53) , and (C55) , we obtain the condition for the 
entire state space to be the capture zone 

H - (w 2 /w 1 ) 2 + (w^/w^) sin 1 (w 2 /w 1 ) - 1} (C56) 

c . The New Geometrical Construction for Optimal Trajectories . 

We consider optimal trajectories in the right half-plane for 
0 y t y = 2 R/w^(tt - s) . Background material is to be found in Appendix 
B. For 0 y x y , we have that <f>(x) = 1 for 0 < s y S , and (C25) 
and (C26) may be written as 

W 1 W 1 
x(t) - R = (i - xw )cos s sin — x + { (£ - xw )sin s - R}cos — x 

Z R Z K 

and 

W 1 W 1 

y(x) = -{ (2 - tw 2 )sin s - R}sin — x + (2 - xw^Jcos s cos — t , 



(C57) 
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which we may write as 



for 0 ^ 


5 < 


S where 


COS S = W^/w^ 


and 0 <; t < x^ 


x(j) - R 




! w i 

COS — T 


. W 1 
sin T 1 




\ 

(JL - xw^)sin s — R 


w 




. "l 
-sin — 
K 


W 1 

T COS T T 1 




(£ - TW2)cos Sy 



The geometric interpretation of (C58) is as follows. To determine a point 
on an optimal trajectory at time t ^minCi/w^, 2R/w^(tt - s)) (we presently 
shall show why addition restriction; see above near (C39) for case of barrier), 
we start at the point x = £ sin s , y = £ cos s where s e UP and move 
to a second point which is a distance tw ^ towards the origin along the 
straight line connecting the first point to the origin. We now rotate this 
second point through an angle (w^/R)t in the negative (clockwise) sense 
about the point x = R , y = 0 . 

We have used the above geometric interpretation (C58) to discover in 
the homicidal chauffeur game a central field of extremals through a point 
on the barrier corresponding to t = £/w^. All "primary solution" extremals 
(see [30] p. 278) pass through this same point. For 0 <_ t <_ > we 

suspect that on any primary trajectory there is no point conjugate to x = 0, 
but we cannot check this by the Jacobi condition, since the strengthened 
Legendre-Clebsch condition is not satisfied, i.e., = 0 (see [9] p. 181, 

also [44] p. 398). The significance of a conjugate point is that the 
primary solution terminates at this locus of focal points to the UP of the 
terminal surface. 

We can, however, investigate the existence of conjugate points by the 
use of our geometrical construction (C58) . Another way of looking at the 
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conjugate point condition is that P^,P 2 C ^ anc ^ ^ the same 

point on an extremal trajectory, i.e., adjacent extremals intersect. By 
consideration of the geometric interpretation of (C58) it is clear that for 
and to yield the same point on an extremal : 



(1) they must lie on the same circle with center x = R, y = 0 , 

(2) it must be possible to obtain P from P^ by an appropriate 
rotation about x = R, y = 0 given by 

- T^w^sin s 1 - R \ / cos — ( t 2 ~ t i^ sin ~jif ^ t 2~ T 1^ (^~T 2 w^)sin s^ - R 

!-' ’h 

i \ w i w i ! 

(2.-t 1 w 2 )cos j \^-sin — ( T 2 " T p cos "r - ( T 2~ T lV \ ^ -T 2 W 2^ C0S s 2 

(C59) 

where T 2 " T 1 > ^ causes a rotation in the negative sense. From (C59) it 
is clearly sufficient that = T 2 = ^^ w 2 * Also if T \ ~ ^2 ’ t ^ en 

necessary that = 2/ • This is easily seen by considering (C59) for 
T ^ = T 2 • It reduces to 



and 



(ft - T 1 w 2 )sin s^ - R = (Jl — x 2 w 2 ) sin s 2 - R , 



(2 - t 1 w 2 )cos s^ = (2 - t^w^cos s 2 



Hence if t s 2 , we must have t.^ = t 2 w 2 = 2 . Since this is also 

sufficient, it suffices to consider t e { t | 2 — tw^ > 0} . Although in 
further research we have not been able to prove or disprove intersection of 
adjacent extremals for t £ 2/w 2 , we have shown the following: 

(a) for P and P 2 £ UP to yield the same point on an extremal 
for = 2 /w 2 , it is necessary that = T 2 ’ 
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and (b) for and e UP and such that 0 <_ <_ S to yield 

the same point on an extremal, we must have * 

To prove (a) we proceed as follows. For P^ and P^ to lie on the 

same circle, by C C 59) we must have 

2 2 
[ (£ - T^^sin s^ — R] + [ (X - t^^cos 

2 2 
[U - x 2 w 2 )sin s 2 - R] + [(i - t 2 w 2 )cos s 2 ] 

which yields the following quadratic equation for w 2 : 

2 2 

W 2 T 2 “ 2t 2^ w 2 “ Rw 2 sin s 2^ ~ [2R£(sin s 2 - sin s^ 

2 2 

- 2t^{w 2 - Rw 2 sin s^ + w 2 T i ] = 0 , 

whose solution is given by 

w 2 t 2 = (1 - R sin s 2 )±/(w 2 t^ - 1 + Rsin s^^ + 2R(sin s 2 - sin s^) (£, - t^w 2 ) . 

Noting that when s^ = s 2 we must have T ± ~ T 2 9 we see ^at tRe m i nus 

sign is extraneous and hence 

W 2 T 2 = ^ ~ Rs i n s 2 ^ + ^ w 2 t 1 ” 1 + Rsin s^^ + 2R(sin s 2 - sin s^) (£ - t^w 2 ). 

(C60) 

Assertion (a) follows from letting = £/w 2 (^60) 

To prove (b) for £ - xw 2 > 0 , we note that for 0 <_ S 2 < s^ <_ tt/2 

we have sin s 2 < sin s^ . Hence using (C60) we see that 

w 2 t 2 - 1 + R sin s 2 < w 2 t i “ R + R sin s 2 



T 1 > T 2 



or 



(C61) 
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As stated above, we suspect that adjacent extremals do not intersect 
for t < £/w 2 , but we have not been able to prove this. We have shown that 
all extremals which terminate on terminal surface for 0 < s < S pass 
through the same point at t = £/w 2 • Hence ? all such trajectories (except 
the barrier s = S ) terminate at this point. 

d . Geometry of the Solution . 

In this section we summarize the results of analysis by drawing a 
picture of the optimal trajectories with the control, <j) , being indicated 
in various regions. The singular subarc, x = 0 , and part of the paths 
leading to it (we have not done this analysis) are also given. As noted above 
the entire state space may or may not be the capture region. From (C56) , 
we let 

£* = R{/l - (w 2 / ^ + (w 2 /w 1 )sin - 1} . (C62) 

Then there are two cases to consider 

(1) i <_ £* barrier meets negative y-axis 

(2) £ > £* entire state space is capture zone. 

In case (1) the Pursuer can only achieve capture for a small portion 
of the state space if the Evader plays properly. This is shown in Figure 
C2, in which only the right barrier is shown. 

In case (2) the entire state space is the capture zone. Optimal tra- 
jectories for this case are shown in Figure 3. There are two further cases. 
For £/w 2 < 2 R(tt - S)/w^ , paths terminating on UP for 0 < s £ S all con- 
verge to point A at x = £/w 2 - This is the case shown in Figure 3. When 
£/w 2 > 2R(tt - S) /w^ , such trajectories meet past point B at the end of the 
barrier, and there is a transition surface between this point and B. As 
w 2 -> 0 , the trajectories approach those shown in Figure B2. 



& 
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Figure C3. 



Part of Optimal Trajectories for Homicidal Chauffeur Game. 
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APPENDIX D. SURVEILLANCE-EVASION GAME. 



In this appendix we derive parts of the solution to the surveil- 
lance-evasion game given by equation (1) of the main text. First, we 
present details of its solution. Next, we discuss our new geometrical 
construction for optimal trajectories in this problem. Finally, we 
discuss aspects of the geometry of the solution and summarize analysis 
results not given in the main text. A more detailed discussion of many 
solution steps is to be found in Appendix B (or Appendix C if they 
occur there ) . 

a. Development of Solution . 

Hamiltonian , H(t,x,p;j),if;) 

The Hamiltonian for the problem given by equation (1) of the 
main text is given by 



S 1 S 1 

[(t ,x,p ;<f> ,i[i) = 1 + P 1 (- y4> + s 2 sin tp) + P 2 (— x< f> ~ s ± + S 2 cos ^ ’ 



whe 



3J 



3J 



re p, (t) = — (t) and p 0 (t) = — (t) are the dual variables and 
1 3x l o y 



J = max min 

V * s 2’*0 



dt = min max 

V* s r*b 



dt. We determine extremal strategies 



from 



max min H(t ,x,p ;<p ,ip) =* 

s 1 ,(f) s 2 ,ip 



and 



max {s 1 [^(-p 1 y + p 2 x)-p 2 ]}, (D2) 



min ts 2 (p 1 sin + p 2 cos ip)}. 



(D3) 



(Dl) 
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It is clear from (D2) (since s^(t) ^ 0) that <|> is given by 

(t) = sgn A (t) , 

where 

A (t ) = -p^ + p 2 x (D4) 

The determination of is more complicated, and we have to use 

results to be established later in this appendix. We parametrically 
represent the terminal surface by u as shown in Figure Dl below. 




Figure Dl. Terminal Surface for Surveillance-evasion Game. 
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Later we will show that (where x = T - t) 



R 1 

A(x) = — {cos u - cos(u + — 4 >x) > , 
9 K 

and 



P 2 ( t ) = -cos(u + — 4 >t ) . 



Substituting the above into (D2), we obtain 



max {s 1 [^ A(x) - p 2 (x ) ] } 
s 1> 4> 

= max{s^ cos u} , 
s i 

and thus we see that (where u refers to termination conditions) 



f u i £ ° r 



s x ( t ) 



= 1 



^ 0 for 



7T TT 

2 < U < 2 



7T 3 

2 < U < ~ TT 



Next, we consider (D3). To minimize with respect to \p it 
suffices to consider 



f(ip) = P-^ sin ip + cos ip 



A necessary condition is that 



and hence 



df A , . , 

3-7 = 0 = p- cos ip - p sin ip, 
dij; 1 l 



tan ip = p 1 /p 2 - 



Thus, to minimize f (ip) we must have 



“Pi 



sin \p = 



/p l + P 2 



/ <'l + P 2 



(D5) 

(D 6 ) 

(D7 ) 

(D8) 



COS if) = 



(D9) 
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with the minimum value being 

min f(ip) = ~/p£ + < 0 , 

which implies that (D3) is given by 

min {s^f (^) > = min {-s^p^ + p^} , 
s 2 ,ip s 2 

and, hence, we see that 



s 2 ( t ) = w 2 . (DIO) 

Boundary Conditions for Dual Variables 

We must have p(t = T) = (p^(T) p 2 (T)) parallel to the normal, 
n, pointing into the state space to the terminal surface (see Figure 
Dl). Now, since n = -sin u l - cos u j", we have 

p^(t = T) = -sin u, p 2 (t = T) = -cos u. (Dll) 

Useable Part of Terminal Surface 
We have that 

. -f ’t 

n = -sin u i - cos u j , 

"t S 1 -* s i 

x = (-— y(p + s 2 sin \p)i 4- (— xcj> - s + s 2 cos ip)j , 

where the terminal surface is parametrically represented by u as 
shown in Figure D1 above. Hence, we have 

x(t = T) = d sin u and y(t = T) = d cos u. 



Hence , 
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n • X(t = T) = 



(- <f>d cos u + 



+ (— <))d sin u - 
K 



and when we recall (see (D9 ) ) that 



sin \p) (-sin u) 

+ s^ cos ty) (-cos u) , 



" P , (t=T) 

sin ij>(t = T) = ■ ■ ■ - = sin u , 



y/ pJ + P 



cos \p(t = T) = 



"P 2 (t=T) 



+ P 



= COS u , 



we obtain from n • X(t = T) ^ 0 that 



n • X (t = T) = — s ^ cos u ^ 0. (D12) 

TT 3 

Recalling (D8) and (DIO), we see that for ~ < u < — tt (D12) is 

7T 7T 

identically satisfied. For - — < u < — , we have 



-w 0 + w, 



cos u ^ 0, 



where we also assume w ? < w^ (otherwise the Evader can always escape 
merely by ’'outrunning’ 1 his pursuer). Hence, the useable part of the 
terminal surface is given by 



UP = {u|u £ u £ 2tt - U where cos U = w 2^ w l y 0 £ U & rr / 2 } . (D13) 

We note that this is the complement of the UP in the homicidal 
chauffer game (see (C 7 ) in Appendix C). 
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The Adjoint Equations and Their Solution 
We have that 



dt 



3H = _J 
3x P 2 R 



d p , 

dt 



3H = _J 
3y P 1 R 



To integrate backwards from time of escape, T, we introduce the 
"backwards time" variable x defined by x = T - t and obtain 



dPl _ s, 

dt P 2 R 



p^(x = 0) = -sin u, 



dp 2 s l 

— = -P x 5- <t> P 2 ( r = 0) = c°s u. 



We combine these equations to obtain a second order differential 



equation for p^ as follows 



d P x dp 2 s 1 

dt z = dr~ F~ 



S 1 2 

= -P X (^ ♦) . 



or 



d2p i ®i 2 

+ (r- ♦> p i ■ °- 



The solution to the above equation is given by 



S 1 S 1 

p^-r) = A cos — <J)T + B sin — <j>T , 
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where the constants, A and B, are determined by 



p^(i = 0) = A = -sin u 



dpi s s 

dT (T = °> = R- ** = F - * p 2 (t = 0) 



cos u 



or 



B = -cos u. 



Hence, after some manipulation, we obtain 



p^Cx) = -sin (u + — 4 >t). 



(D14) 



Similarly, it may be shown that 



P 2 (t) = —cos (u + — <j>T). 



(D15) 



Taking note of equation (D9), we also obtain 



S 1 S 1 

sin ^ = sin(u + — - <f>i) and cos ip = cos (u + — 
R R 



(D16) 



Taking account of (D8), we obtain from the above (where u refers to 



angle at which escape occurs, see Figure Dl) 



w. 



7T 3 2 

for U ^ u < - and — tt < u £ 2 tt - U where cos U = — 
2 2 w^ 



and 0 £ U £ 



TT 



W, 



p (t) = -sin (u + — <t>T ) , p (t ) = —cos (u + — (pr ) , 



and 



W 1 W 1 
sin \p = sin(u + — (pr) , cos ip = cos (u 4* — 4 >t) , 



(D17 ) 



( 18 ) 
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, _ 7T 3 

and for — < u < y tt 



P 1 (t) = -sin u , p 2 (t) = -cos u, 



(D19) 



and 



sin = sin u , cos ip = cos u. 



(D20) 



Solution to the State Equations 

In the "backwards time' 1 we have 



- s 2 sinCu+^x) 

d S 1 S 1 

dr = - R~ <t ’ X + S 1 “ s 2 cos ^ u + r~ <t>T ) 



x(t = 0) = d sin u, 



y(T = 0) = d cos u. 



The above equations have the same form as the state equations for the 
homicidal chauffer problem (see Appendix C). Hence, their solution is 
given by 



and 



® 1 D S 1 

x(x) = (d - xs 2 )sin(u + ][_ <P T ) + t(1 “ c °s — <J>x), 



S 1 R S 1 

y ( t) = (d - ts 2 )cos(u + — 4>t) + — sin — (fix. 



(D21) 



(D22) 



Taking account of (D8) , we obtain from the above (where u refers to 

angle at which escape occurs, see Figure Dl) 

7i 3 W 2 

for U £ u < — and -r-ir<u£2ir-U where cos U = — 

Z Z w^ 

and 0 <; U 

w w 

x (t) = (d - xw 2 >sin(u + 4 it) + ^(1 - cos — 4>x) , (D23) 



and 



w w 

y(t) = (d - xw 2 )cos(u + —=■ 4t) + ^ sin <pT , 



(D24) 
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, r 7T 3 

and for — < u < — tt 

x(t) = (d - Tw^sin u , y(x) = (d - xw 2 )cos u. (D25) 

We note that equations (D23) and (D24) (and also (D17) and (D18)) 
hold from the terminal surface (x = 0) until a transition surface is 
reached or the trajectory terminates. 

Equation of the Barrier 

From the symmetry of problem, we consider the right barrier, 
which terminates on the terminal surface with u = U, where 0 £ ^ < 
and 

cos tl = w^/w^. (D26) 

For u = U and 0 £ x £ x^, we have 4>(x) = 1 (we show this in the 
next section) and the barrier is given by 

for 0 £ x £ x 

W 1 w l 

x (x ) = (d - xw 2 )sin(U + — x) + R(1 - cos — x), (D27) 

and 

w w 

y (x) = (d - xw 2 )cos(U + ^ T ) + R sin — x . (D28) 

We now consider 

w w 

R cos — x = R cos (— x + U - U) 

R K 

w W 

= R cos (==■ t + U )cos U + R sin (— t + U) sin U . 

R t 



Using the above and (D26) , we obtain 



NJ j =3 
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W 1 W 2 

R cos — x = R — 
R w 



w. 



w„ 



cos t 4- U ) + — / w^ - w^ s in x + U ) , 

R 1 2 K 

which we combine with (D27) to obtain 

W W WWW 

x (t)-R - -R — =■ cos(rpx + U)+{ [d - — / w* - wjr ] - (R — )rp- t }sin t + U) 

W^ K W^ 1 Z W K R 

Similarly 



W 2 W 1 

y (x) = (R — )sin(— t + u)+{ 

W^ K. 



R /— 2 Z 

— / wf - w$ 

w 1 2 



w w w 

w 9 1 w 1 

_ (R ^ )c °3 ( a T + u ) 



Recalling (C30) and (C31), we see that the above are the equations of 
an involute to a circle of radius R w^/w^ and with center x = R,y = 0. 
We note that the involute winds in for 



d - R /l -(v^/w^) 2 " W 2 X ^ 0 



(D31) 



Transition Surfaces 
We recall that 



4> (x ) = sgn A(x), 



(D4) 



where A(x) = -p^y + p^ x 
Recalling that 



x(t = 0) = d sin u , y(x = 0) = d cos u, 



and 



p^(i = 0) = -sin u , p 2 ( T = 0) = -cos u , 



we see that 



A (t = 0) = 0. 



(D32) 



Also , 



dA 

dx 



dx 



dy dp 2 
y - p i dr + XT 



x + p 



dx 
2 dx 



(D29) 



(D30) 
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into which we substitute the state and adjoint equations to obtain 

dA S 1 

= = s i sin ( u + ♦ 'O with A(t = 0) = 0. 

Integration of the above yields 



R ° i 

A(t) = T {cos u - cos(u 4- — 4 >t )}, 
<p R 



(D 3 3 ) 



Recalling (D8) and noting that for the escape paths terminating with 

77 3 

~2 < u < — 7T the Pursuer uses s^(t) = 0, we see that in this range 
there is no transition surface since A(t) = 0 for all time. Hence, 
we consider only paths for which there is a transition surface, i.e., 

U £ u < —■ and ~ tt < u 2 tt - U. In this range of u by (D8) we have 
S 1 = W 1 anC * (^33) becomes 



w. 



R 1 

A(t) = — {cos u - cos(u + — 
© K 



(D34) 



Thus, for 0 £ t (where we determine below), 

+1 for U ^ u < ~ 



4> (t ) = 



-1 for ~ it < u ^ 2tt - U, 



(D35) 



where U is given by (D26) . By the symmetry of the problem we concen- 



trate on the right half-plane. Consequently, for ^ ^ u < 



we 



determine as follows: it is the first time that A(t) = 0 after 

t = 0. This happens when 






cos u = cos (u + — t ) 
K 1 



which is precisely when 



W l 

2 TT - U = u + 
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Hence , for U 



7T 

^ U < “, 



we have 



2R 

T = (7T 

1 w i 



u). 



(D36) 



It appears as though all trajectories except the barrier (u = U) 
terminate before this time, however, because of intersection of 
adjacent extremals. Hence, equation (D36) becomes 



T , = — (tt - U), (D37) 

1 w i 

where U is given by (D26). For t > we show later that A(i) < 0 

and hence 4>(t) - -1 . We finally note that (D37) holds only if the 
barrier doesn’t terminate before this time is reached. 



Termination of the Barrier 

A fundamental difference between the homicidal chauffer game 
and the surveillance-evasion game is that in the latter the barrier may 
terminate abruptly before the transition given by (D37) occurs. We 
recall (see Appendix C) that in the homicidal chauffer game the barrier 
terminated when 4 >(t) changed from +1 to -1. In the surveillance- 
evasion game we shall see that for 

d < R {/l - (w 2 /w 1 ) z + 2(w 2 /w 1 X it -U) , (D38) 

the barrier terminates at 

T 2 = (d - R sin U)/w (D39) 

We have investigated many aspects of this phenomena and will detail our 
findings here. 
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What led us to our findings was a statement by Dobbie [14] which 

we found to be incorrect. Let us consider Figure 5 of the main text. 

Dobbie ([14] p. 176) states that the (right) barrier can be tangent to 

the line through 0 and D* (if d is small enough). This was found 

to be incorrect. Let be a unit vector parallel to line connecting 

0 and D f and v tobeaunit vector orthogonal to v . Thus 

^ i 

—v t -y — >- 

v^ = sin U i - cos U j , 

v^ = cos U i 4* sin U j . (D40) 



We show below that there is a transition from 4> — 1 to <p - -1, i.e,. 

A(t) changes from positive to negative, at = 2R(tt - U)/w^. Now 

we will show that we always have 



dX 



dx 



T = T, 



v i = °> 



(D41) 



i.e., the tangent to the barrier is orthogonal to the line through 0 



dX 



dx 



and D ? at x = x^. Also, 

dX 

when we have that — = 0. Hence the barrier can never be tangent to 

dx 



• v = 0 only for d - x,w* = R sin U, 
x=x^ 2 J 12 



this line. We should note that for x = x^ our new geometric construc- 
tion (discussed below) shows that the point (x (x ^ ) , y (x ^ ) ) of the 
barrier always lies on the line through 0 and D f (if we ignore 
termination) . 

To prove (D41) we consider (1) of the main text at x^ = 2R/w^(tt - U) 



and 



dx 



dx 



dx 



x=x. 



X = T, 



W 1 

— (d - x w )cos U - w sin 2 U + w^sin U, 
R 1 2 i 2 



— (d - x w )sin U + w cos 2 U - w cos U, 
R 1 2 1 2 



(D42) 
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where we have also used (D27) and (D28) . Now 



dX 

dx 



X = X- 



v i = 



dx 



dx 



sin U - 



T — X _ 



dy_ 

dx 



^ x=x. 



cos u , 



which use of (D42) and some manipulation shows to be equal to w^ 
-w^ cos U = 0, and hence (D41) is proved. 

The second statement we prove similarly 



dX 



dx 



X “X „ 



V 2 = 



dx 



dT J 

J X = T - 



COS + 



d£ 

dx 



sm 



U 



x=x. 



or 



dX 



i dx ; 

^ j T — 



w. 



V, = — {d - XjW - R sin U}, 



x=x . 



2 R 



(043) 



where we have used (D42) and simplified. Thus, 



dX 



IdxJ 



• v^ = 0 
x=x^ 2 



for d - x^w^ = R sin U • If we substitute this latter condition into 
(D42) we find that 



dx 



l dx; 



x=x. 



= sin U {-w cos U + = 0, 



and 



dZ 

dT) 



x=x 

1 



cos U {w^ cos U - w 2 ) = 0. 



Let us now discuss how when (D38) holds the barrier has a cusp 
at t = T 2 given by (D 39 ) . We obtained several clues by considering 
our new geometric interpretation of the solution to the state equations, 
which we now discuss. The equations of the barrier (D27) and (D28) may 



be written as 
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W 1 w i 

x(t)- R = (d - tw )cos U sin — t + { (D - tw )sin U - R}cos t, 



and 



w w 

y(x) = -{ (d - tw )sin U - R}sin — t + (d - tw )cos h cos — t, 



which we may write as 



for 0 £ t £ t. 



w., 



w. 



x(t)- R 


/ 


. y( T > j 





COS — T sm — T 

K R 



w„ 



(d - Tw^)sin U - R 



(d - tw^cos U 



(D44 ) 



Let us consider Figure 5 of the main text. To determine a point on the 

right barrier at time t £ 2R/w^(tt - U ) , we start at point A at the 

boundary of the detection region and move to a point which is a distance 

tw^ away from A along the straight line 0A. We now rotate this 

W 1 

point through an angle — t in the negative (clockwise) sense about 

R 

the point x = R,y = 0. 

Some further remarks seem appropriate. When t = t^ we rotate 
through an angle 2 tt - 2h in the negative sense or an angle of 2U 
in the positive sense. Hence, any point on the straight line through 
0 and A is carried to the line through 0 and D*. Hence, our remark 
about a point of the barrier being on OD T at t = t^. We also see 
the suggestion of anomalous behavior if the point moving along OA 
passes the point of tangency to the circle of radius Rw^/w^ and center 
x = R,y = 0. If this occurs the geometry of the situation tells us that 
the point of the barrier for t = t^ 



lies on OD ’ above D* toward 
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the boundary of the detection region. As the point moves along OA 
toward 0 before rotation we see that 

(a) when it is above the point of tangency to the circle, 
the resultant point (after rotation) is moving toward 
this circle, 

(b) when it is at the point of tangency, the resultant point 
is on the circle of radius Rw^/w^, 

and (c) when it moves past the point of tangency for x < x^, 
the resultant point moves away from the circle. 

Again we note that this is because the construction rotates the point 
about x = R,y = 0. 

Algebraically, we can see the above approach and recession by 
considering (D44) from which we can obtain the square of the distance, 
denoted by D(x), of a point on the barrier at x from x = R,y = 0 

D(x) = (x(x)- R) 2 + y 2 (x) = (d - xw^ - R) 2 + 2R(1 - sin U) (d - xw^) . 

Hence, for the minimum of D(x), we must necessarily have - 0, 

which yields 

x 2 = (d - R sin U)^. (D45) 

This value of x does indeed yield a minimum since = 2 ^ 2 ) 2 > 0. 

Hence, we suggest that the barrier has a cusp at such a point, 
if it occurs. Let us further note that such behavior can occur on any 
optimal trajectory terminating on the boundary of the detection region 
for U < u < “*. Thus, it appears that a trajectory terminating at a 
point P on the boundary (if d is right) similarly approaches and 
receeds from the circle to which OP is tangent. We have not had time 
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to explore this further. Let us note that this doesn't occur for 

TT 

u = ~. We note that for to occur before x^ we must have 

d - R sin U < 2R ” U)/w 

or 



d < R{ sin U + 2 w^tt - U)/w 1 

which by (D26) is precisely the condition given by (D38). 

Now that we are suspicious that if (1)38) holds, the barrier 
has a cusp in it if it doesn't terminate at x^» let us prove the 
existence of the cusp algebraically before we plot the curve of the 
barrier. Differentiating (D23) , we obtain 



dx 

dx 



W W W « 

-w 0 sin(U + — x)+ — {(d - xw 0 )cos (U + — x) + R sin — x) ♦ 
K 



R 



R 



R 



(D46) 



Noting that 

w w 

-w sin(U + — x)+ w sin(— - x + U - U) = 

Z R IK 

w w w x 

-w„ sin(li + — x)+ w. sin(U 4- — x)cos U - w cos (— x + U)sin U 

2 R 1 R IK 

W 1 

= -w cos(— x + ^) sin U, 
i R 



where we have used (D26) , we see that (D46) is equal to 

A W 1 

^ = -w, cos(U+ -i x) {sin U - (d - x w )/R}. (D47) 

dx 1 R 4 
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Similarly 




(D48) 



and hence 




(D49) 



dy 

Thus, we see that is continuous at 

dx 



t ~ t 2 ~ (d - R sin^O/w^. 
and change sign as x passes 



but both and ^ - 0 at x = x. 

dx dx 2 



through x 2 * Hence, the barrier curve y = y(x) has a cusp at x - x^- 



Thus, the barrier must terminate (see Appendix B for discussion) and 
(D38) and (D39) have been proved. 

Another way to see this is to consider (D29) and (D30) . From the 
discussion of the involute in Appendix C, we see that the involute 
winds in for 



doesn’t occur before the involute touches the circle with radius 
Rw^/w^, the involute starts to unwind. But for this to hold, (D38) 
follows immediately. We note that the involute unwinds for 



Figure D2 shows the cusp which would occur if barrier didn’t 
terminate. The values of parameters which were used to calculate the 
curve are shown on the figure. 



, ^ w i 

d - R/l -(w /w )* - (R-)ex) * 0. 

Z 1 W n K 



Therefore if 



x = 2 (tt - U) , 

-L W_ 



{ [d - R/l ~(w 2 /w 1 ) z ] 



W w 

(R < °- 
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= 20 


R = 


20 


M 


w 2 = 10 


d = 


10 u 





Figure D2. Occurrence of Cusp if Barrier Does Not Terminate. 
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Continuation of Solution Past Transition Point 

It may be shown that when barrier does not terminate (see (D38)) 

other extremals intersect before x = x^ and hence terminate, so we 
the 

consider justi(right) barrier. From (D17), we have 



P 1 (x 1 ) = sin U , P 2 ( t 1 ) = " cos U • 



From (D27) and (D28) , we obtain 



x(t 1 ) = (d - x^w^) (-sin U) + R(1 - cos 2 U) , 

y (x 1 ) = (d - cos ^ “ R sin 2 ^ . (50) 

Later, we show that <j>(x) = -1 for x > x^. We assume this for now. 
The adjoint equations for x > x^ are 

dp i 

XT' = -p 2 W ]/ R = sin U , 

d p 2 

— = P^/R P 2 (t x ) = -cos U . 



The solution to the above equations is (for x > x^) 

W 1 

p^x) = sin (U + — (x - x )) = -sin ip , 

W 1 

P 2 <.t) = -cos (ll + — (x - x^)) = -cos <p. (51) 



The state equations for x > x^ 



are 
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It = “ y + W 2 si "( U+ ]T (T " T l )} ’ 

d W 1 W 1 

= R - X + W 1 ' W 2 C ° S(U + F" (T " T l )} ’ 



with initial conditions given by (D50). A rather laborious computation 
(we omit the details) yields the solution to the above as (for x > x^) 



x (x ) = -(d - xw 2 )sin(U + 




(T - T^) 



W W 

+ R{ 2 cos — (x - x^)- 1 - cos(2 'l + — (x - T^))}, 



and 



y (x ) = (d - xw )cos(U + -^-(x - x )) 



w w 

+ R{2 sin — (x - x^)- sin(2 U + — (x - x^))}. 



(52) 



We may write equations (D52) as (for x > 



x (x )+R 




W 1 1 

cos — (x-x^) U -sin — (x-x^) 




— 


w. 




y(x) 




1 

sin — (x-x ) 


W 1 

COS — (x-T 1 ) > 



or using (D50) 



x (x)+R 




r 

cos 


i 

r (T ‘ T i ) 


1 'l 

-sin — (x-x^) 




- 




w. 


w 


. y(T). 




^sin 


1 

F (T ' T i ) 


cos — (t-t^., 



T l } 



(d-xw^) (-sin U) + 2R - R cos 2^ 



(d-Ti^^cos U - R sin 2U 



x(t^)+ W 2 (x-T 1 )cOS (-J - U) + R 

„ y(x 1 )-w 2 (x-x 1 )sin(y - U) 



(D53) 



(D54) 



110 



The most important aspects of the geometric interpretation of 
(D54) are that rotation is in the positive (counter-clockwise) sense 
about the point x = -R,y = 0 and the involute unwinds. That (D52) 
does indeed trace out an involute may be seen by writing it in the 
equivalent form 




/ ! w 2 w w w 

1 “(~) - d} + (R -^) (x ]cos(U + ^(t - T. )) 

W^ W, _L 



w r 



+ R — sin 
w„ 



w 

in (U + —=■(-[• - tj), 



(D55) 



which is the involute to a circle of radius R — and with center 

w i 

x = -R,y = 0. 



Shape of the Surveillance Region 

When a surveillance region does exist, it may take on one of two 
shapes depending on whether or not the barrier terminates by intersect- 
ing the negative y-axis. These two possibilities are shown in Figure 
5 and 6 in the main text where further details are given. We develop 
here the condition for the barrier to look like that shown in Figure 6. 

d ^ R{/1 — (w 2 / w^ ) ^ +(w 2 /w 1 ) (~tt-U)+1}, 



(D56) 



Ill 



which we may also write as 

d ^ R{Vl -(w 2 /w 1 ) 2 +(w 2 /w 1 )sin 1 (-w 2 /w 1 )+ 1}, (D57) 

where 

it £ sin 1 (-w 2 /w^) £ 3/2 tt. 

We consider Figure D3 (which isn't drawn to scale) and recall the 
relationship between two normals to the involute of a circle given by 
(C32) (see Appendix C) . In Figure D3, we have 

£ 1 + Z 2 = Z 3 9 (D58) 

i.e., the difference in the length of the normals is equal to the distance 
on the perimeter of the circle (evolute) between the points of tangency. 
When the barrier intersects the negative y-axis we have 

R £ Z . (D59) 

We also have 



= d - Z. or 

3 4 



= d - R sin U, (D60) 

and 



^2 = R w /w ol , (D61) 

where 



2 (tt - U) = a + 6 or a = 2ir-2U-Or/2-U) > 



or 



a = 3/2 tt - U. 



(D62) 
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y 



i 




i 



Figure D3. 



Determination of Tangency Condition 
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Combining (D58) , 



which by use of 



(D59), (D60), (D61), and (D62) , yields 
R + R(w 2 /w ] )(3/2 tt - U) £ d - R sin ll , 
(D26) is seen to be (D56) . Noting that 

“1 h 

<3/2. -U), 



we see that 



(U63) 



W 1 l 2 

si n (~ = - cos U = - w 2 /w L , 

and hence 

& 2 =(Rw 2 /w 1 )sin 1 (-w 2 /w 1 ) , 

which leads to (D57) . 

Absence of Singular Solution 

Unlike the problems considered in Appendices B and C, there does 
not appear to be a singular solution to this problem. 

Another Way to Determine UP 

We have seen (see Appendix B) that the useable part of the terminal 
surface may be determined from X • n <> 0, where n is a unit normal 
vector to the terminal surface and points into the state space. Another 
criterion used by Isaacs ([30] p. 239) in a capture game is that the 
UP is determined by ^ where + y 2 = r 2 . For the escape 

problem at hand, this condition becomes 
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(r r) ^ 0. (D64) 

r=d 

[We note that this condition is examined only on the terminal surface 
and that optimal strategies are not determined by the rate of change 
of range as one might infer from Dobbie [14] pp. 175-176 (see [30] 
pp. 205-206) . ] 

Using (1) and (DIO), we obtain from (D64) 

w (x sin ip + y cos \p)- s y ^ 0, (D65) 

2 1 

which becomes for t = 0 and using (D20) 



dw 2 (sin 2 u + cos 2 u) ^ s^d cos u. 



whence letting s^ = w^ 



W 2 ~ W 1 COS u ^ (D66) 

• 

which is the same result as from X • n <> 0. 

b . The New Geometrical Construction for Optimal Trajectories . 

We consider optimal trajectories in the right half-plane for 
0 <> t <> = 2R/w^(tt - u) and U ^ u < tt/2, cos ^ = w^/w^. Background 

material is to be found in Appendix B. For 0 <> x ^ x^, we have that 
4 >(t) = 1, and by (D23) and (D24) we have 

x(t) 

and 

y (t) 



W 1 W 1 
R = (d - twJcos u sin — x + { (d - xw )sin u - R}cos — x, 
Z K Z K 



w i w i 

- {(d - iw„)sin u - R}sin — t + (d - tw )cos u cos — 
Z K Z K 



T, 



(D67) 
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which we may write as 

for U £. u < it/ 2 where cos U 



W 2^ W 1 3n< ^ 



0 s; t <. 



x(t)- R 

. y(x) , 




(D68) 



Equation (D68) appears in the main text as (3). The geometric interpre- 
tation of these equations follows equation (3) in the main text* 

Considering the geometric interpretation of (D68) , we see that 
the involute winds in, i.e. , radius of curvature decreases with 
increasing t, for 0 ^ t £ t^. Further geometric investigations 
have shown that neighboring extremals intersect (see Figure 3). This 
means that on an extremal there is a point conjugate to t = 0 and 
that the trajectory terminates here (except for the barrier). Time has 
not permitted this to be more fully investigated. 



c . Geometry of the Solution . 

This topic is discussed in section Hid of the main text. 
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APPENDIX E 

SOME ALLOCATION OF SEARCH EFFORT RESULTS 
BY THE PONTRYAGIN MAXIMUM PRINCIPLE 

A major problem in Naval warfare is the search for targets at sea. 

Hence, the optimum allocation of search effort is of interest in developing 
tactics. We show in this appendix that some well-known results in the 
optimum allocation of search effort may be more easily obtained by appli- 
cation of the Pontryagin maximum principle [43] and the "sharpness" of 
the results extended slightly. We begin by reviewing briefly the litera- 
ture. 

Koopman wrote the first major work on search theory [35] , which was 
for a long time classified (until May 10, 1961). It still remains a 
major work, especially remarkable for containing many of the concepts for 
research being performed 20 years later. Som examples; formulation of non- 
linear programming problem for searching ocean areas, Bayesian approach to 
sequential search. Later, Professor Koopman published some of these results 
in the open literature [36], [37], [38]. In [38], Koopman solved the problem 
for a continuous, one-dimensional model of the optimum distribution of search 
density when the conditional probability of detecting the target is exponen- 
tial. However, it should be noted that Chapter 3 of [35] contains much 
material not in [38]. For example, in [35] Koopman first solves a discrete 
problem of searching effort, discusses the physical interpretation of various 
quantities and the structure of the solution, and then extends this to the 
case where targets are continuously distributed (two-dimensional) . 

Charnes and Cooper [10] extended [38] by formulating a non-linear 
programming model for searching discrete alternatives. They solved this 



117 



model by applying the Kuhn-Tucker conditions. De Guenin [13] extended 
Koopman r s results [38] by considering the optimum distribution of search 
density for a general conditional probability of target detection (he 
assumed this to be a concave function of search density). 

We now derive the solution to Koopman's problem [38] and de Guenin *s 
[13] by the maximum principle. The reader is directed to these papers for 
model formulation. Even though Koopman's problem is a special case of 
de Guenin* s, there still appears to be methodological value in considering 
it first. 

a . Koopman f s Problem . 

The problem studied by Koopman may be stated as to maximize target 
detection probability when there is a restriction on the total amount of 
search effort available. Mathematically, the problem may be stated as 



maximize 



p(x){1 - e"* 00 } 



dx 



subject to: 



► oo 

<(> (x) dx = $ , 

j — 0O 



and (j) (x) >_ 0 , (El) 

where 

p(x)dx = Prob[ target located between x and x + dx ] , 

1 - (X) = Prob [detect target with effort <f> (x) | target located at x ] , 

c|) (x) is search density and defines the distribution of search effort, 

and is total search effort. 
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We note that 



p(x) dx = 1 and p(x) >_ 0 



We now consider the following equivalent optimal control problem: 



maximize 



p(x){l - e dx , 



subject to: dy/dx = <J> (x) , 

where $ (x) > 0 , 



and 



y (x = — «>) = 0 and y (x = -h°°) = $ 



(E2) 



The Hamiltonian for this problem is 

H (x , y , A , <f> ) = p(x){l - e ^ + A<j> (x) > 



(E3) 



where X is the dual variable corresponding to the state equation. 



Defining J* as equal to max 



p(x){l - e ^ X ^}dx , we see that 



X = 3J*/3y < 0 

since y(x) is cumulative effort (y(x) = 



•x 

<J) (x) dx) and by expending 

J —00 



more effort than is optimal we can do nothing but reduce the optimum target 
detection probability. By the condition 



dX = _ 9JH 
dx 3y U , 

we see that X (x) is a constant. 

The optimum distribution of search effort is found by maximizing the 
Hamiltonian with respect to the control variable (j) , which is the search 
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density. To facilitate a later argument, we let A = -u where p > 0 . 

Then, the Hamiltonian is maximized by 

8H/3$ = p(x)e~* (x) — ji = 0 for <j>(x) > 0 
or 

9H/9<f> < 0 for (j) (x) = 0 

2 2 i / \ 

Since 3 H/ 3 cj> = -p(x)e < 0 for all x and <j>(x) < 00 , we see that 

sufficient conditions for a global maximum are satisfied, i.e., H (4> ) is 
concave. Thus, we see that the optimum distribution of search effort is 
determined as follows: 

(a) for p(x) < y, <f>(x) = 0 , since then p(x)e ^ ^ < y or 3H/3<f> < 0 , 

(b) for p (x ) >_ y, 4> (x) = in(p(x)/y) , since p(x)e~^ X ^ = y yields 

3H/3<|> = 0 (E4) 

We determine y as follows. Define ft = {x|p(x) y} . Then y is chosen 

so that 



<* A CO 

£n(p(x)/y)dx = <f)(x)dx = $ 

ft -°° 



When the appropriate sufficient conditions (see [9] pp. 181-182) are checked, 
i.e. strengthened Weierstrass, strengthened Legendre, and Jacobi, it is found 
that (E4) is both necessary and sufficient, 
b . De Guenin’s Problem . 

Here, we consider a more general conditional probability of target 
detection. The problem is 
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maximize 

<f> 



p (x)h(<f> (x) )dx , 



subject to: 



<f> (x)dx = $ 



and 



<f>(x) i o , 



(E5) 



where p(x) , (j> (x) , and $ are defined as before and 

h((f>(x)) = Prob[detect target with effort <$> (x) | target is at x ] . 

De Guenin further assumes that h((f>) is concave with h ! ((j> = 0) 0 , 

i.e., h l (<t>) = dh/d<j> is marginal return of conditional detection probability 
with search effort. Diminishing returns may be stated as h"(<f>) < 0 with 
dh/d<j> (cf> = 0 ) > 0 and dh/d<j) (<J> = ») = o . It is noted that the condition 
h”(cf)) < 0 for all <j> implies that the inverse of h r (cf> ) , i.e. h 1 ^ , is 
well-defined. 

Again, we consider an equivalent optimal control problem: 

f 00 

maximize p(x)h(4>(x)) dx , 

<j) ^ -00 

subject to: dy/dx = <\> (x) , 

where (f> (x) >_ 0 

and y (x = -°°) = 0 and y (x = -f°°) = $ (E6) 



The Hamiltonian for this problem is 



H(x ,y , A ,(f>) = p (x)h(<(> (x) ) + A<J> (x) , 



(E7) 
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where, as before, the dual variable, A , will turn out to be negative 

A = 3J*/3y < 0 



As before, 

dA/dx = -3H/3y = 0 , 

so the dual variable is a constant, which we, for convenience, set equal to 
-y where y > 0 . Maximization of the Hamiltonian with respect to the 
control variable <j> is determined by 

3H/3<J) = p(x)h f (4>) - y = 0 for (J> (x) > 0 , 
or 

3H/9<j) < 0 for (x ) = 0 , 

which is sufficient for a global maximum, since 

3 H/3<f>2 = p(x)h n ((f)) < 0 for all x and <|>(x) . 

We determine the desired search density, <J> , as follows: 

3 H 

(a) for p(x)h ! ((f) = 0) < y, 4 (x) = 0 , since then — < 0 , 

H *-0 

(b) for p(x)h'(il! = 0) 2. b > 4>(x) = h 1 *(u/p(x)) , since h'(ij)) = u/p(x) 

3 H 

yields — = 0 , (E8) 

where y is determined similarly to the previous case. It is easily shown 
that (E8) are both necessary and sufficient for the optimum distribution 
of search effort. Thus, we have shown that de Guenin f s results are also 



sufficient for the determination of optimum search effort. 
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c • Extensions . 

Here, we mention some extensions of either analysis or models to the 
above. It can be shown that the above results may be developed strictly 
within the framework of the classical calculus of variations (and in a 
different way than reported in the literature) but this analysis has not 
been completely documented at this time. Besides the trivial extension to 
h = h(x,c})(x)) , already noted by de Guenin, we may extend the maximum 
principle approach to some cases where h(<f)) is not concave. 
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